Method for displaying contents on basis of smart desktop and smart terminal

ABSTRACT

A content displaying method according to an embodiment includes: displaying windows allocated to users on a touch panel; displaying pieces of content input from the users on the windows; and displaying thumbnails on the windows, wherein the displaying of the thumbnails includes displaying pieces of content displayed on all windows in the thumbnails.

TECHNICAL FIELD

The present disclosure relates to smart terminal technology, and moreparticularly, to a content displaying method based on a smart desktopand a smart desktop terminal.

BACKGROUND ART

With developments in technology, smart desktop terminals have becomepart of people's daily life and work. A multi-party conference usingsmart desktop terminals may be a familiar working scenario. During sucha conference, it is common practice to write on smart desktops displayedby the smart desktop terminals, e.g., taking notes, writing a solution,or drawing a picture.

However, when a plurality of users use a smart desktop terminaltogether, a method of displaying content on a smart desktop becomes animportant issue.

DESCRIPTION OF EMBODIMENTS Technical Problem

Provided is a content interacting method based on a smart desktop.

Solution to Problem

A content displaying method according to an embodiment includesdisplaying windows allocated to users on a touch panel, displayingpieces of content input from the users on the windows, and displayingthumbnails on the windows, wherein the displaying of the thumbnailsincludes displaying pieces of content displayed on all windows in thethumbnails.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a flowchart illustrating a content displaying method based ona smart desktop, according to some embodiments of the presentdisclosure.

FIG. 2 is a flowchart illustrating a user authentication methodaccording to some embodiments of the present disclosure.

FIG. 3 is a flowchart illustrating a content displaying method for asmart desktop including an associated device, according to someembodiments of the present disclosure.

FIG. 4 is a schematic diagram illustrating a multi-party conferenceaccording to some embodiments of the present disclosure.

FIG. 5 is schematic diagram illustrating a display of a thumbnailaccording to Embodiment 1 of the present disclosure.

FIG. 6 is a schematic diagram illustrating a display of the thumbnailaccording to Embodiment 1 of the present disclosure.

FIG. 7 is a schematic diagram illustrating a display of a subjectsummarization, according to Embodiment 2 of the present disclosure.

FIG. 8 is a schematic diagram illustrating a display of a usersummarization, according to Embodiment 3 of the present disclosure.

FIG. 9 is a schematic diagram illustrating sharing with a designateduser, according to Embodiment 4 of the present disclosure.

FIG. 10 is a schematic diagram illustrating sharing based on adesignated position, according to Embodiment 5 of the presentdisclosure.

FIG. 11 is a schematic diagram illustrating a sharing region accordingto Embodiment 6 of the present disclosure.

FIG. 12 is a schematic diagram illustrating a writing conflict accordingto Embodiment 7 of the present disclosure.

FIG. 13 is a schematic diagram illustrating a first embodiment of adisplay after writing conflict processing, according to Embodiment 7 ofthe present disclosure.

FIG. 14 is a schematic diagram illustrating a second embodiment of adisplay after writing conflict processing, according to Embodiment 7 ofthe present disclosure.

FIG. 15 is a schematic diagram illustrating a third embodiment of adisplay after writing conflict processing, according to Embodiment 7 ofthe present disclosure.

FIG. 16 is a schematic diagram illustrating occlusion according toEmbodiment 8 of the present disclosure.

FIG. 17 is a schematic diagram illustrating a first embodiment of adisplay after occlusion processing, according to Embodiment 8 of thepresent disclosure.

FIG. 18 is a schematic diagram illustrating a second embodiment of adisplay after occlusion processing, according to Embodiment 8 of thepresent disclosure.

FIG. 19 is a schematic diagram illustrating recording resource conflictprocessing according to Embodiment 9 of the present disclosure.

FIG. 20A is a schematic diagram illustrating time reminding resourceconflict processing when alarm clocks have the same alarm time,according to Embodiment 9 of the present disclosure.

FIG. 20B is a schematic diagram illustrating time reminding resourceconflict processing when alarm clocks have different alarm times,according to Embodiment 9 of the present disclosure.

FIG. 21 is a schematic diagram illustrating a first embodiment ofproviding a prompt, according to Embodiment 10 of the presentdisclosure.

FIG. 22 is a schematic diagram illustrating a second embodiment ofproviding a prompt, according to Embodiment 10 of the presentdisclosure.

FIG. 23 is a schematic diagram illustrating a third embodiment ofproviding a prompt, according to Embodiment 10 of the presentdisclosure.

FIG. 24 is a schematic diagram illustrating a smart whiteboard accordingto Embodiment 11 of the present disclosure.

FIG. 25 is a schematic diagram illustrating an exemplary structure of acontent displaying apparatus for a smart desktop, according to someembodiments of the present disclosure.

FIG. 26 is a schematic diagram illustrating an exemplary structure of auser authentication apparatus according to some embodiments of thepresent disclosure.

FIG. 27 is a schematic diagram illustrating an exemplary structure of asmart desktop terminal according to some embodiments of the presentdisclosure.

FIG. 28 is a schematic diagram for explaining a smart desktop accordingto an embodiment.

FIG. 29 is a diagram for explaining changing of a position and an angleof a window, according to an embodiment.

FIG. 30 is a diagram for explaining a method of adjusting contentaccording to an embodiment.

FIG. 31 is a flowchart for explaining a method of displaying a thumbnailaccording to an embodiment.

FIG. 32 is a flowchart for explaining a method of displaying a windowaccording to an embodiment.

BEST MODE

A content displaying method according to an embodiment includes:displaying windows allocated to users on a touch panel, displayingpieces of content input from the users on the windows, and displayingthumbnails on the windows, wherein the displaying of the thumbnailsincludes displaying pieces of content displayed on all windows in thethumbnails.

MODE OF DISCLOSURE

Hereinafter, the present disclosure will be described in detail withreference to the accompanying drawings in order to enable one ofordinary skill in the art to easily embody and practice the presentdisclosure. However, the present disclosure is not limited to examplesdisclosed below, but may be implemented in various forms. Also, parts inthe drawings unrelated to the detailed description are omitted to ensureclarity of the present disclosure. Like reference numerals in thedrawings denote like elements.

The terms used in the present disclosure are those general termscurrently widely used in the art in consideration of functions in thepresent disclosure, but the terms may vary according to the intention ofone of ordinary skill in the art, precedents, or new technology in theart. Accordingly, the terms used herein should not be interpreted onlybased on names of the terms but should be interpreted based on meaningsof the terms and the whole context of the present disclosure.

It will be understood that, although the terms ‘first’, ‘second’, etc.may be used herein to describe various elements, these elements shouldnot be limited by these terms. These terms are only used to distinguishone element from another

Also, the terms used herein are merely used to describe exemplaryembodiments, and are not intended to limit the present disclosure. Anexpression used in the singular encompasses the expression of theplural, unless it has a clearly different meaning in the context.Throughout the specification, it will be understood that when an elementis referred to as being “connected” to another element, it may be“directly connected” to the other element or “electrically connected” tothe other element with intervening elements therebetween. It will befurther understood that when a part “includes” or “comprises” anelement, unless otherwise defined, the part may further include otherelements, not excluding the other elements

The use of the terms “a”, “an”, and “the” and similar referents in thecontext of describing the disclosure (especially in the context of thefollowing claims) is to be construed to cover both the singular and theplural. Also, the steps of all methods described herein may be performedin any suitable order unless otherwise indicated herein or otherwiseclearly contradicted by context. The present disclosure is not limitedto the described order of the steps.

The phrase “in some embodiments” or “in an embodiment” shown in variousparts of the specification does not necessarily indicate the sameembodiment.

Some embodiments of the present disclosure may be described in terms offunctional block components and various processing steps. Some or all ofsuch functional blocks may be realized by any number of hardware and/orsoftware components configured to perform specified functions. Forexample, functional blocks of the present disclosure may employ one ormore microprocessors, or various circuit components which carry outpredetermined functions. For example, the functional blocks of thepresent disclosure may be implemented with any programming or scriptinglanguage. The functional blocks may be implemented in algorithms thatare executed on one or more processors. Furthermore, the presentdisclosure could employ any number of conventional techniques forelectronics configuration, signal processing, and/or data processing.The words “mechanism”, “element”, “means”, and “configuration” may beused broadly and are not limited to mechanical or physical embodiments.

Furthermore, the connecting lines or connectors shown in the variousfigures are intended to represent exemplary functional relationshipsand/or physical or logical couplings between various elements. It shouldbe noted that many alternative or additional functional relationships,physical connections, or logical connections may be present in apractical device.

The present disclosure will now be described more fully with referenceto the accompanying drawings.

FIG. 1 is a flowchart illustrating a content displaying method based ona smart desktop according to some embodiments of the present disclosure.

As shown in FIG. 1, the method includes the following.

In step 101, a displaying instruction for content of a smart desktop isdetected.

In step 102, the content is displayed on the smart desktop based on adesignated displaying manner.

In some embodiments, the designated displaying manner includes at leastone of:

displaying manner (a): displaying the content in a thumbnail based on arelative position of the content on the smart desktop; and

displaying manner (b): displaying the content and/or a summarization ofthe content based on a user attribute and/or a text attribute associatedwith the content.

The smart desktop in the embodiments of the present disclosure may beimplemented by an independent device or a plurality of interconnectedassociated devices.

If the smart desktop includes a plurality of interconnected associateddevices, the displaying of the content in the thumbnail based on therelative position of the content on the smart desktop (a) may include:detecting a relative position of each associated device, and displayingthe content of each associated device in the thumbnail based on therelative position of the associated device.

The detecting of the relative position of each associated device mayinclude at least one of the following.

(1) Determining the relative position of each associated device based ona network signal detecting result of each associated device.

For example, the relative position of each associated device may bedetermined based on a combined detecting result of one or more of awireless communication signal, an infrared communication signal, aBluetooth communication signal, and a wireless local area network signalof the associated device.

(2) Determining the relative position of each associated device based ona relative position setting value.

For example, each associated device may set a relative position settingvalue and exchange the relative position setting value with otherassociated devices. Therefore, a position topology of the associateddevices may be determined based on each relative position setting value.As such, the relative position of each associated device may bedetermined.

The above description provides a general embodiment for detecting therelative position of each associated device. One of ordinary skill inthe art would know that the above description is merely an example andis not used for restricting the protection scope of the presentdisclosure.

In some embodiments, the displaying of the content in the thumbnailbased on the relative position of the content on the smart desktop inthe displaying manner (a) may include at least one of the following.

(1) Determining a viewing direction of a user and adjusting a displayingdirection of the content in the thumbnail according to the viewingdirection of the user.

In particular, the determining of the viewing direction of the user mayinclude: determining the viewing direction of the user according to ahandwriting characteristic of handwritten content of the user; and/or,determining the viewing direction of the user according to a collecteduser image.

(2) Compressing a redundant area in the thumbnail.

In some embodiments, the compressing of the redundant area in thethumbnail may include:

dividing the content into one or more content blocks according to atleast one of a semantic correlation, a position correlation, a timesequence correlation, and a user correlation; and expanding one or morecontent blocks to remove the redundant area from among the contentblocks.

(3) Adjusting a font of the content in the thumbnail.

In particular, a font of each content block in the thumbnail may beadjusted to a suitable font, such that the content block is exactlyfilled up with the content as far as possible and the user is able tosee the content as clearly as possible. The font of each content blockmay be adjusted. The adjusting scale of the font of each content blockmay be different or the same.

(4) Displaying the summarization of the content in the thumbnail.

(5) Hiding private information in the content, etc.

For example, private information in the content or in the summarizationof the content may be determined based on a user-defined manner or atext analyzing manner. The hiding may include: not displaying theprivate information or replacing the private information by a privateidentifier, etc.

In some embodiments, the displaying of the content in the thumbnailaccording to the relative position of the content on the smart desktopin the displaying manner (a) may further include at least one of thefollowing.

(1) Detecting an operation for selecting content in the thumbnail, andproviding an enlarged display for the selected content in the thumbnailand/or an adjacent area of the thumbnail.

Through providing the enlarged display for the selected content, theuser may easily view the selected content.

(2) Prompting the user in the thumbnail when detecting an update of thecontent of the smart desktop, and updating the content in the thumbnailaccording to a user instruction.

Through prompting the user of the update of the content in thethumbnail, the user is able to know the update of the content of thesmart desktop in time. Further, the content may be updated in thethumbnail based on an instruction initiatively triggered by the user.

(3) Updating the content in the thumbnail when detecting an update ofthe content of the smart desktop.

When it is detected that there is an update for the content of the smartdesktop, it is also possible to initiatively update the content of thesmart desktop without prompting the user or a user instruction.

(4) Receiving a content editing instruction and editing the content inthe thumbnail based on the content editing instruction.

In some embodiments, the displaying of the content and/or thesummarization of the content based on the user attribute and/or textattribute associated with the content in the displaying manner (b) mayinclude: hiding the private information in the content and/or thesummarization of the content. In some embodiments, the method mayfurther include at least one of the following.

(1) Receiving a content viewing instruction, and displaying contentassociated with the summarization based on the content viewinginstruction.

(2) Receiving a summarization length changing instruction, and changinga length of the summarization based on the summarization length changinginstruction.

(3) Receiving a searching instruction, and searching in the contentand/or the summarization of the content based on the searchinginstruction.

In some embodiments, the content includes content to be shared. Thedesignated displaying manner in step 102 includes:

determining a user receiving the content to be shared and/or adisplaying position of the content to be shared; and displaying thecontent on the smart desktop based on the determined user and/ordisplaying position. In view of the above, through designating the userreceiving the content to be shared and/or the displaying position of thecontent to be shared, the content to be shared may be displayed to theuser receiving the content, or may be displayed in any displayingposition.

In some embodiments, the content to be shared may be determined via thefollowing manner:

designating existing content as the content to be shared; or,designating content which is written in real time as the content to beshared.

In some embodiments, the method may further include:

adjusting the displaying direction of the content to be shared accordingto a viewing direction of the user receiving the content to be shared.

Herein, the determining of the viewing direction of the user mayinclude: determining the viewing direction of the user based on ahandwriting characteristic of handwritten content of the user; ordetermining the viewing direction of the user based on a collected userimage (e.g., a detected user sitting image), etc.

In some embodiments, the content includes content to be shared. Thedesignated displaying manner in step 102 may include: setting a sharingregion on the smart desktop; receiving a content sharing determininginstruction; and in response to the content sharing determininginstruction, copying the content to be shared to the sharing region. Inview of the above, according to the embodiments of the presentdisclosure, the content shared by the user may be copied to thedesignated sharing region. As such, other users may conveniently obtainthe shared content.

In some embodiments, the method may further include at least one of thefollowing.

(1) Receiving a sharing region rotating instruction, and in response tothe sharing region rotating instruction, rotating the sharing region.

A user desiring to obtain the shared content may not be near adisplaying position of the shared content. At this time, throughinitiatively rotating the sharing region, the user desiring to obtainthe shared content may be close to the shared content conveniently.

(2) Receiving a sharing region rotation stop instruction, and inresponse to the sharing region rotation stop instruction, stopping therotation of the sharing region.

When the content to be shared is rotated to nearby the user desiring toobtain the content, the user may stop the rotation of the sharing regionby triggering the sharing region rotation stop instruction.

(3) Receiving a sharing region content obtaining instruction, and inresponse to the sharing region content obtaining instruction, copyingcorresponding content in the sharing region.

If the content to be shared is stopped nearby the user desiring toobtain the content, the user may copy the content to be shared to hisdisplaying area in the sharing region, so as to obtain the contentshared by other users.

In some embodiments, the content includes: existing content which has aposition conflict with content currently being written.

The detecting of the displaying instruction for the content of the smartdesktop in step 101 includes: detecting the existing content which hasthe position conflict with the content currently being written, andtriggering the displaying instruction for the existing content whendetecting the existing content.

The designated displaying manner in step 102 includes: temporarilyhiding the existing content.

According to the embodiments of the present disclosure, through hidingthe existing content having the position conflict, it is possible tocontinue the current handwriting.

In some embodiments, the method further includes:

after an input of the content currently being written on the desktop isfinished, determining a display resuming manner based on the semanticcorrelation between the existing content and the current currently beingwritten; wherein the display resuming manner includes at least one of:resuming the display after moving a position of the existing contentand/or the content currently being written; and resuming the display byrearranging a layout of the existing content and/or the contentcurrently being written.

In some embodiments, the content includes: content covered by anoccluding object on the smart desktop.

The detecting of the displaying instruction for the content of the smartdesktop in step 101 includes: detecting the occluding object on thesmart desktop, and, when the content of the smart desktop is covered bythe occluding object, triggering the displaying instruction for thecovered content.

The designated displaying manner in step 102 includes: rearranging alayout of the covered content.

In some embodiments, the rearranging of the layout of the coveredcontent includes: rearranging the layout of the covered contentaccording to at least one of: a character continuity, a text linecontinuity, a paragraph continuity, a semantic correlation, an imagecontinuity, and a correlation between an image and text of the coveredcontent.

In view of the above, the present disclosure provides a solution for theconflict between the newly input content and the existing content andfor the covered content, so as to solve the conflict between the newlyinput content and the existing content and ensure a normal display ofthe covered content.

In some embodiments, the method further includes:

determining prompt information for the content currently being writtenbased on the existing written content and displaying the promptinformation.

In view of this, according to the embodiments of the present disclosure,prompt information may be displayed, which may intelligently prompt theuser of content or a summarization that is not written by the user buthas been written by other users and intelligently correct the contenthaving been written by the user through analyzing the content written byall users, which improves the writing experience of the user, increasesthe efficiency for accomplishing the same task by multiple users, andavoids repeated conceiving and writing by the multiple users.

In some embodiments, the determining of the prompt information for thecontent currently being written according to the existing writtencontent includes at least one of:

(1) retrieving a keyword from the content currently being written, anddetermining the prompt information for the content currently beingwritten according to a keyword retrieved from the existing writtencontent if a similarity degree between the keyword retrieved from thecontent currently being written and the keyword retrieved from theexisting written content is higher than a threshold,

(2) retrieving a keyword from the content currently being written,obtaining a context keyword corresponding to the keyword retrieved fromthe content currently being written according to a keyword and a contextkeyword retrieved from the existing written content, and determining theprompt information for the content currently being written according tothe obtained context keyword,

(3) retrieving a context keyword from the content currently beingwritten, obtaining a keyword corresponding to the context keywordretrieved from the content currently being written according to akeyword and a context keyword retrieved from the existing writtencontent, and determining the prompt information for the contentcurrently being written according to the obtained keyword, and

(4) retrieving a summarization for the existing written content from theexisting written content, retrieving a summarization for the contentcurrently being written from the content currently being written,comparing the summarization for the existing written content with thesummarization for the content currently being written, and determiningthe prompt information for the content currently being written accordingto a compared result.

In some embodiments, the content includes time reminding information.

The detecting of the displaying instruction for the content of the smartdesktop in step 101 includes: detecting a reminding time set by theuser, and triggering the displaying instruction for the content when thereminding time set by the user arrives.

The designated displaying manner in step 102 includes: if each userusing the smart desktop sets the same the reminding time, when thereminding time arrives, playing the time reminding information via atleast one of: a voice, an icon, and text; or

if each user using the smart desktop sets a different reminding time,when the reminding time arrives, playing the time reminding informationvia at least one of: an icon and text.

In view of the above, the embodiments of the present disclosure realizeparallelized scheduling of time reminding resources of the smart desktopfor different users, and thus a resource utilization rate is increased.

In conclusion, for multiple user-invoking requests for the same resourceof the smart desktop, the present disclosure provides a solution whichis able to solve a conflict between the multiple user invoking requestsand ensure normal and reasonable invoking of the smart desktop.

In some embodiments, the method may further include:

starting recording after receiving a recording start instruction fromthe user;

stopping recording after receiving recording stop instructions from allusers; and

generating a recording file for each user according to the recordingstart instruction and recording stop instruction of each user.

Therefore, the embodiments of the present disclosure also realizeparallelized invoking of recording resources of the smart desktop formultiple users, and thus increase a resource utilization rate.

In some embodiments, the method may further include:

receiving written content, generating a user characteristic based on ahandwriting characteristic of the written content and a writingassistant characteristic, and saving the user characteristic; and

receiving content written by a user to be authenticated, generating auser characteristic to be authenticated based on a handwritingcharacteristic of the content written by the user to be authenticatedand a writing assistant characteristic, and determining thatauthentication succeeds if the user characteristic to be authenticatedmatches the saved user characteristic.

In view of the above, the present disclosure realizes userauthentication based on a new user authentication method. The user doesnot need to write content used for authentication in advance. Instead,the authentication may be performed on the user during a free writingprocedure.

In some embodiments, the method may further include:

establishing a connection relationship between the smart desktop and anassociated device, and exchanging content of the smart desktop and/orcontent of the associated device between the smart desktop and theassociated device based on the connection relationship.

In some embodiments, the exchanging of the content of the smart desktopwith the associated device may include the following:

(1) Determining content to be distributed in the content of the smartdesktop, and determining from among associated devices the associateddevice that is to receive the content to be distributed; and in responseto a content distribution instruction, transmitting the content to bedistributed to the associated device that is to receive the content tobe distributed.

For example, an application scenario may include: in class, a teacheruses a smart whiteboard and students use the associated devices. Adetailed behavior may include: the teacher distributes tasks to thestudents.

(2) Obtaining editing information for the content provided by thedesignated associated device, and transmitting the editing informationto the designated associated device.

For example, an application scenario may include: in class, the teacheruses the smart whiteboard and the students use the associated devices. Adetailed behavior may include: the teacher inspects and grades thestudents' homework.

(3) Determining content to be shared on the smart desktop andtransmitting the content to be shared to the designated associateddevice.

For example, an application scenario may include: in class, the teacheruses the smart whiteboard and the students use the associated devices.The detailed behavior may include: the teacher shares teaching content.

(4) Receiving editing information of the associated device for thecontent shared by the smart desktop, and displaying the received editinginformation.

For example, an application scenario may include: in class, the teacheruses the smart whiteboard and the students use the associated devices.The detailed behavior may include: the teacher receives the students'notes.

(5) Determining an area of the smart desktop covered by the associateddevice, and transmitting content in the area of the smart desktop to theassociated device.

For example, an application scenario may include: a plurality of personsare around the smart desktop and each one holds an associated devicesuch as a cell phone. The detailed behavior may include: putting theassociated device such as the cell phone on content of the smart desktopthat is to be saved. At this time, the smart desktop senses an areacovered by the associated device and then transmits the content in thearea to an application in the associated device such as the cell phonefor storage via manners such as near-field communication (NFC) orwireless fidelity (Wi-Fi).

(6) Transmitting designated content on the smart desktop to thedesignated associated device.

For example, an application scenario may include: a plurality of usersare around the smart desktop and each one hold an associated device suchas a cell phone. The detailed action may include: a user performs aspecific action (e.g., clicking a content transmitting button). Afterdetecting the user's action, the smart desktop transmits designatedcontent on the smart desktop to the designated associated device.

(7) Receiving the content transmitted by the associated device,determining the area covered by the associated device, and displayingthe received content in the determined area.

For example, an application scenario may include: a plurality of usersare around the smart desktop and each one holds an associated devicesuch as a cell phone. The detailed action may include: a user puts theassociated device such as the cell phone on an area of the smart desktopwhere he expects to display content of the cell phone. At this time, thesmart desktop senses that the area is covered by the associated device.The content may be transmitted via manners such as NFC or Wi-Fi anddisplayed in the area.

In some embodiments, the smart desktop may include associated deviceshaving a connection relationship.

The displaying of the content on the smart desktop based on thedesignated displaying manner in step 102 includes:

obtaining content of the associated device according to the connectionrelationship, and displaying the obtained content based on thedesignated displaying manner.

In the prior art, an authentication method based on content written by auser mainly includes: a user writes preset signature content in advance.A device records and saves a handwriting characteristic (e.g., a writingcharacteristic) of the signature content. When it is required toauthenticate the user, the user writes the same signature content again.The device matches the handwriting characteristic of the contentcurrently being written and the saved handwriting characteristic so asto determine the identity of the user. However, in this authenticationmethod, the user has to register handwritten content in advance and hasto write fixed signature content during authentication. The user may notbe able to write freely and this may bring trouble to the user.

According to an embodiment, the present disclosure further provides auser authentication method. Based on the user authentication method, auser does not need to register handwritten content in advance and userauthentication may be performed during a free writing procedure of theuser.

FIG. 2 is a flowchart illustrating a user authentication methodaccording to some embodiments of the present disclosure.

As shown in FIG. 2, the method includes the following.

In step 201, written content is received, a user characteristic isgenerated based on a handwriting characteristic of the written contentand a writing assistant characteristic, and the user characteristic issaved.

Herein, a device does not require a user to write preset signaturecontent and allows the user to write freely. At this time, the devicedoes not know the identity of the user. While receiving the writtencontent input by the user in real time, the device records thehandwriting characteristic (such as a pressure, a speed, anacceleration, a stroke characteristic, etc.) of the written content andalso records the writing assistant characteristic (such as positioncoordinates, time information, a writing direction, and a text linedirection for the writing, etc.). Then, the device may generate the usercharacteristic of the current user through a characteristic retrievingalgorithm and saves the user characteristic in the device.

In step 202, content written by a user to be authenticated is received,a user characteristic to be authenticated is generated based on ahandwriting characteristic of the content written by the user to beauthenticated and a writing assistant characteristic, and if the usercharacteristic to be authenticated matches the saved usercharacteristic, authentication succeeds.

During continuous writing of the user to be authenticated, the usercharacteristic to be authenticated is generated based on the handwritingcharacteristic (e.g., a pressure, a speed, an acceleration, a strokecharacteristic, etc.) of the content written by the user to beauthenticated and the writing assistant characteristic (e.g., positioncoordinates, time information, a writing direction, a text linedirection for the writing, etc.). The user characteristic to beauthenticated is compared with the user characteristic saved in thedevice. During the matching, the device may determine the identity ofthe user based on various aspects such as a handwriting characteristic,writing position coordinates, a time continuity, a writing direction, atext line direction, a writing area, and written content.

If the user characteristic to be authenticated matches the saved usercharacteristic, the authentication of the user identity succeeds.Otherwise, if the user characteristic to be authenticated does not matchany saved user characteristic, the user characteristic to beauthenticated is saved and a new user identifier is created for thecurrent user.

According to an embodiment, the present disclosure further provides acontent displaying method for a smart desktop including an associateddevice.

FIG. 3 is a flowchart illustrating a content displaying method for asmart desktop including an associated device according to someembodiments of the present disclosure.

As shown in FIG. 3, the method includes the following.

In step 301, a connection relationship with an associated device isestablished.

In step 302, content of a smart desktop and/or content of the associateddevice are exchanged with the associated device based on the connectionrelationship.

In particular, the exchanging of the content of the smart desktop and/orthe content of the associated device may include at least one of thefollowing:

(1) Determining content to be distributed in the content of the smartdesktop, and determining from among associated devices the associateddevice which is to receive the content to be distributed; and inresponse to a content distribution instruction, transmitting the contentto be distributed to the associated device that is to receive thecontent to be distributed.

(2) Obtaining editing information for the content provided by thedesignated associated device, and transmitting the editing informationto the designated associated device.

(3) Determining content to be shared on the smart desktop andtransmitting the content to be shared to the designated associateddevice; and receiving editing information of the associated device forthe content shared by the smart desktop, and displaying the receivedediting information.

(4) Determining an area of the smart desktop covered by the associateddevice, and transmitting content in the area of the smart desktop to theassociated device.

(5) Transmitting designated content on the smart desktop to thedesignated associated device.

(6) Receiving the content transmitted by the associated device,determining the area covered by the associated device, and displayingthe received content in the determined area.

Hereinafter, the above solution of the present disclosure is describedin detail with reference to the accompanying embodiments.

FIG. 4 is a schematic diagram illustrating a multi-party conference. Asshown in FIG. 4, multiple users sit around a smart desktop and start themeeting. Each user may write content on the smart desktop in an areaclose to him following his writing habit.

In the prior art, viewing requirements of users in different directionsmay not be met. For example, when multiple users use a smart desktop towrite content, it is inconvenient to view the content written by otherusers due to their different sitting positions and different directions.For example, the users sitting face to face have opposite viewingdirections. Thus, it is difficult for them to view content written bythe other user.

Embodiment 1

Hereinafter, a solution is described in detail: a displaying instructionfor content of a smart desktop is detected; and the content is displayedon the smart desktop based on a designated displaying manner. Thedesignated displaying manner may include: displaying the content in athumbnail based on a relative position of the content on the smartdesktop.

As shown in FIG. 4, it is often necessary to write content, e.g., takingnotes, writing a solution, drawing a picture, etc. When multiple userswrite content by using a smart desktop, it is inconvenient to view thecontent written by other users due to their different sitting positionsand different directions.

Scenario: a user A is hosting a brain storm. Other users have writtentheir thoughts on respective positions of the smart desktop. The user Awants to view solutions of all users.

Solution: a thumbnail based on a relative position of content

Based on the present disclosure, a thumbnail may be provided accordingto an operation of the user A. The providing manner may include at leastone of the following:

(1) Voice Operation

The user A provides a voice, e.g., saying “I want to see a thumbnail”.At this time, the smart desktop displays the thumbnail in a blank areain front of the user A. If an area in front of the user A is filled up,content in front of the user A in an area with the same size as thethumbnail may be hidden to display the thumbnail.

(2) Physical Button Operation

The user A may call out the thumbnail by pressing a certain button onthe smart desktop.

(3) Touch Screen Operation

The user A may call out the thumbnail by touching a screen, e.g., doubleclicking the screen. At this time, the smart desktop displays thethumbnail on a position where the user touches the screen. If there iscontent on this position, the content in an area with the same size asthe thumbnail may be temporarily hidden to display the thumbnail.

(4) Gesture Operation

The user A provides a gesture action, e.g., swiping a hand. At thistime, the smart desktop captures the gesture action of the user A via acamera and displays the thumbnail in a blank area in front of the userA. If an area in front of the user A is filled up, content in front ofthe user A in an area with the same size as the thumbnail may be hiddento display the thumbnail.

(5) External Controller Operation

The user operates an external controller, e.g., pressing a button of astylus pen. At this time, the smart desktop displays the thumbnail on aposition pointed by the stylus pen. If there is content on thisposition, content in an area with the same size as the thumbnail may betemporarily hidden to display the thumbnail.

FIG. 5 is a schematic diagram illustrating a display of a thumbnailaccording to Embodiment 1 of the present disclosure. The left part ofFIG. 5 illustrates an actual display of the smart desktop, whereindifferent users have different writing positions and directions. If auser wants to view written content of other users, a thumbnail may becalled out as shown in the right part of FIG. 5.

The user may have various configurations for his thumbnail. Theconfigurations may include the following.

(1) Determine Whether Non-Handwritten Content is Displayed in theThumbnail

As a viewer, when the user A confirms to display merely handwrittencontent, merely handwritten content of users of the smart desktop (allusers or users other than the viewer) is displayed in the thumbnail,whereas non-handwritten content such as text, an image, slides, and avideo is not displayed.

(2) Determine Whether Content of the User Himself is Displayed in theThumbnail

As a viewer, when the user A confirms to display merely content ofhimself, the content of the user A (the viewer) is displayed in thethumbnail. If the user A confirms not to display the content of himself,the content of the user A is not displayed in the thumbnail.

(3) Determine whether pieces of content of different users aredifferentiated by colors

According to an embodiment, it is possible to recognize different usersaccording to their handwriting, so as to obtain content written by thedifferent users.

When the user A confirms to differentiate pieces of content of thedifferent users by colors, the pieces of content written by thedifferent users may be differentiated by different colors in thethumbnail on the smart desktop, i.e., the pieces of content written bythe different users have different handwriting colors. For example, auser B has written in the top-left corner and bottom-left corner of thesmart desktop. At this time, a system recognizes that pieces of contentin the two areas are both written by the user B. When the user A viewsthe thumbnail, the pieces of content in the two areas are displayed inthe same color which is different from colors of content written byother users. Even when the user B has written in two areas by usingdifferent colors, pieces of content in the two areas are presented inthe same color in the thumbnail called out by the user A.

If the user A confirms to display all pieces of content in theiroriginal colors, the pieces of content in the thumbnail of the smartdesktop remain in original written colors. For example, the user Bwrites in both the top-left corner and bottom-left corner of the smartdesktop, wherein content in the top-left corner is written in black andcontent in the bottom-left corner is written in blue. At this time, inthe thumbnail viewed by the user A, the content in the top-left corneris still displayed in black and the content in the bottom-left corner isstill displayed in blue.

According to an embodiment, user information (e.g., a user name, a userbust image, etc.) may be displayed in the thumbnail of a region adjacentto content written by each user.

According to an embodiment, user information (e.g., a user name, a userhead image, etc.) may be displayed in the thumbnail adjacent to contentwritten by each user. Also, when the user A selects content in thethumbnail (e.g., in a closed region), user information corresponding tothe content selected by the user A may be displayed.

(4) Select a Size and a Position for the Thumbnail

When it is confirmed to automatically determine a size and a position ofthe thumbnail, an initial size of the thumbnail is a standard size setin the system and a displaying position is determined automatically. Thesmart desktop preferably displays the thumbnail in a blank area in adisplaying area corresponding to the user. If the blank area in thecorresponding displaying area is slightly smaller than the standard sizeof the thumbnail set in the system (e.g., a difference between a size ofthe blank area in the corresponding displaying area and the standardsize set for the thumbnail in the system is smaller than a preset areathreshold), the size of the thumbnail may be slightly reduced to fit forthe blank area in the corresponding displaying area. However, if theblank area in the corresponding displaying area is far smaller than thestandard size set for the thumbnail in the system (e.g., the differencebetween the size of the blank area in the corresponding displaying areaand the standard size set for the thumbnail in the system is not smallerthan the preset area threshold), the user has to proceed to thefollowing next level options:

i. Covering Mode (Covering Existing Content)

If the user does not need to view content written by himself whileviewing the thumbnail, this option may be selected. At this time, thesmart desktop displays the thumbnail in the corresponding displayingarea of the user and temporarily hides content covered by the thumbnail.

ii. Inserting Mode (Inserting into Content)

If the user needs to view the content written by himself while viewingthe thumbnail, this option may be selected. At this time, the smartdesktop inserts the thumbnail at the end of the corresponding displayingarea of the user. Subsequently, the content handwritten by the user isdisplayed around the thumbnail. If a displaying space is not enough forthe handwritten content of the user, a solution such as changing alayout and dividing into multiple pages may be utilized.

If it is confirmed to manually determine the size and position of thethumbnail, the user determines the size and position of the thumbnailvia a touch screen gesture. For example, the touch desktop detects thatthe user touches the screen on at least two positions at the same time.The smart desktop determines the position and size according to thedetected touching positions of the user. For example, when the usertouches the bottom-left of the screen with the left index finger and thetop-right of the screen with the right index finger at the same time,the position pressed by the left index finger will be the bottom-leftcorner of the thumbnail and the position pressed by the right indexfinger will be the top-right corner of the thumbnail. At this time, ifthere is content on the position determined by the user, it is possibleto proceed to the following next level options:

i. Covering Mode (Cover Existing Content); ii. Inserting Mode (Insertinto the Content)

(5) Select a Desired Area for Generating the Thumbnail

In a thumbnail frame, a reduced-size screenshot of the current screen isdisplayed. The 100% whole screen is selected in default. At this time,the user may select an area for forming the thumbnail by dragging thescreenshot or the thumbnail frame. If a border of the area selected bythe user cuts complete content, the border may be automatically expandedto include the complete content. When the user is interested in merelysome content in the screen, the user may select such an area via thismanner. The system will generate the thumbnail for the area selected bythe user.

According to an embodiment, a configuration interface may be providedafter the user calls out the thumbnail. The user may set as describedabove on the configuration interface. The smart desktop displays thecorresponding thumbnail according to the user's setting. Optionally, theuser may also preset a displaying manner of the thumbnail in advance.After the user calls out the thumbnail, the smart desktop displays thecorresponding thumbnail according to the displaying manner preset by theuser.

In addition, when the thumbnail is displayed, the user may adjust thedisplaying manner of the thumbnail in real time. The smart desktop maydynamically adjust the displayed thumbnail. For example, the user setsin advance not to display content written by himself in the thumbnail.Therefore, the thumbnail initially presented by the smart desktop doesnot include the content written by the user. If the user sets to displaythe content written by himself via a preset operating manner (e.g., avoice, a screen gesture, etc.) while viewing the thumbnail, the smartdesktop dynamically adjusts the thumbnail in real time to present thecontent written by the user in the thumbnail.

According to an embodiment, pieces of content to be displayed in thethumbnail may be displayed in the thumbnail according to relativepositions of the pieces of content on the smart desktop. In other words,a position relationship between the pieces of content in the thumbnailis kept consistent with that on the smart desktop.

After the setting is finished, the system displays the thumbnail in thethumbnail frame. The displaying of the content in the thumbnail includesthe following.

(1) Block Division

The system may determine which pieces of content are highly correlated(e.g., a paragraph may not be segmented) and takes such pieces ofcontent as one block based on the following several correlations:

(a) Content correlation: it is determined whether a part of content hasa close semantic relationship with another part of content based ontechniques such as handwriting recognition, semantic analysis, symboldetection, and text summarization. For example, a symbol mark or revisedcontent for a paragraph is highly correlated with content of thisparagraph.

(b) Position correlation: it is determined whether two parts ofhandwritten content are highly correlated based on a positionrelationship of the two parts. For example, two paragraphs with similarpositions and aligned on left and right sides are determined to behighly correlated.

(c) Time sequence correlation: a time sequence of written content isdetected. For example, content which is written continuously has a hightime sequence correlation.

(d) User correlation: a handwriting characteristic and a font size ofcontent are detected. If two parts are written by the same user, a usercorrelation is high.

The smart desktop may divide the content on the smart desktop into oneor more content blocks based on at least one of the above correlations,wherein the correlations are considered according to a followingpriority order: a content correlation, a position correlation, a timesequence correlation, and a user correlation.

After the block division, the divided blocks may be rearranged,including block expansion and content adjustment.

(2) Block Expansion

After the content blocks are determined, each content block representsmeaningful and indivisible independent content. The system may determineboundaries of all blocks. A remaining part on the screen is a blank areaincluding no content. Since the blank area is a redundant area in thethumbnail and is meaningless, the redundant area between the contentblocks may be removed by expanding one or more content blocks, so as toimprove the viewing effect of the thumbnail and save screen resources.The block expansion manner includes: (a) Detecting block informationentropy: for each block, information entropy of the block is determinedbased on factors such as a block size, an average font size, the numberof characters, an image size, a graph content density, etc. The largerthe information entropy, the more the content and the denser thecontent. For example, if a large area includes merely several very largecharacters, the information entropy is relatively small. (b) Determiningan expansion rate: an expansion rate (may be a negative value, i.e., theblock contracts) of each block is determined according to theinformation entropy of the block. At this time, the boundaries of allblocks will expand at the respective expansion rate, e.g., expandingoutward 5 mm per second. (c) Expansion stop condition: when boundariesof two blocks encounter each other during the expansion, the expansionof the two blocks stops, which does not affect the expansion of otherblocks, until the expansion of all blocks stops. For a contractingblock, the contraction is stopped when the block contracts to athreshold set by the system (relevant to factors such as a content fontsize).

FIG. 6 is a schematic diagram illustrating a display of the thumbnailaccording to Embodiment 1 of the present disclosure. As shown in FIG. 6,a “{circle around (2)} compressed blank area” illustrates a result ofblock expansion.

The block expansion provided by the present disclosure is an optionalstep. The objective of the block expansion is to compress the blankarea. Besides the above solution for realizing the block expansion, thesmart desktop may also compress the blank area via other manners, e.g.,the user selects designated content and instructs via an operation(e.g., a multi-finger expansion operation) the smart desktop to providean expanded display. After detecting the above operation, the smartdesktop provides an expanded display for the content designated by theuser, and ends the expansion operation after detecting that the userinstructs to end the operation.

(3) Content Adjustment

After the boundary of each block is determined, the content may beadjusted as follows: (1) direction adjustment: the smart desktopdetermines a viewing direction of the viewer, and adjusts a displayingdirection of the content in the thumbnail according to the viewingdirection of the viewer, e.g., adjusting a displaying direction ofcontent of each block in the thumbnail to be consistent with the viewingdirection of the viewer, such that the viewer may view content writtenby other users conveniently; wherein the viewing direction of the viewermay be determined via an image of the viewer collected by a camera,e.g., directly determining the viewing direction via the image of theviewer, or obtaining a sitting posture of the viewer via the image ofthe viewer and obtaining the viewing direction based on the sittingposture of the viewer. The viewing direction may also be determined viaa handwriting characteristic of the viewer, or may be designated by theviewer. As shown in FIG. 6, a “{circle around (1)} content directionchange” illustrates a result after direction adjustment.

(2) Font adjustment: the smart desktop adjusts a font of the content ofeach block in the thumbnail to a proper font to let the content exactlyfill out the block and the user may see the handwritten content as clearas possible. The smart desktop performs the font adjustment on eachblock. The font adjustment scale of each block may be different.Therefore, it is also referred to as non-proportional scaling. As shownin FIG. 6, “{circle around (3)} non-proportional scaling” illustrates aresult after font adjustment.

(3) Summarization display: the smart desktop may automatically generatea summarization based on the content in the thumbnail. For example, if afont size of a particular block is smaller than a threshold (relevant toa size of the thumbnail and a distance between the user and thethumbnail, the threshold represents a font size that the user may justsee clearly), the smart desktop displays the content of the block in theform of a summarization. It is also possible to control the generationof the summarization by the user. For example, the user may adjust thecontent in real time during a viewing procedure of the thumbnail, e.g.,setting that a particular part of the content is displayed in the formof a summarization via an operation such as a voice or a screen gesture.After detecting the user's operation, the smart desktop generates asummarization for the part of the content selected by the user andpresents the summarization of the part of the content in the thumbnail.The user may also view the complete content corresponding to thesummarization.

In addition, the thumbnail may further have the following functions:

1. Privacy Setting

The user may set a part of his content as private content via mannerssuch as selecting a closed region by a gesture and hope that suchprivate content is invisible for other users. In addition, the user mayselect to which users that he does not want to expose the privatecontent. At this time, when the smart desktop detects that the selecteduser opens the thumbnail, the private content of the user is notpresented in the block, i.e., the smart desktop hides the privatecontent of the user in the thumbnail.

The hiding of the private information in the thumbnail includes: notdisplaying the private information, or replacing the private informationby a private identifier.

In particular, the private identifier, e.g., “private content”, may bedisplayed in an area corresponding to the private information in thethumbnail. Or, it is possible to display no information and compress thecorresponding area to display other content, so as to save a displayingspace effectively.

As shown in FIG. 6, “{circle around (4)} hide private content”illustrates a result after privacy setting.

In addition, according to an embodiment, the smart desktop mayautomatically detect whether the content of the user is private content.For example, if content such as figures in a personal account, a username, or a password is detected, it is determined that the correspondingcontent may be private content.

2. Enlarged Viewing

When the user A wants to view detailed original content of a part (e.g.,a summarization) in the thumbnail, the user A may select the block viamanners such as clicking the touch screen. At this time, the smartdesktop detects the user's operation selecting the content, and displaysthe selected content in an enlarged manner in the thumbnail and/or in anarea around the thumbnail, i.e. the content of the selected block isdisplayed in the thumbnail or around the thumbnail along with a font anda layout of the original content. If there is much content, viewingmanners such as left-right and up-down scrolling may be provided. Asshown in FIG. 6, “{circle around (5)} zoom in” illustrates a result ofenlarged viewing.

3. Content Update

(1) When the user A views the thumbnail, if another user modifies orcomplements the content, the smart desktop prompts the user in thethumbnail when detecting that the content is updated. For example, ifdetermining that the other user's modification is temporarily finishedvia a manner such as a time sequence (e.g., the writing has been stoppedfor a while), the smart desktop displays an update prompt in the blockof the content on the thumbnail of the user A in the form of an icon orgradient background color. The user A may provide a content instructionby clicking the touch screen on the content of the thumbnail, and thesmart desktop updates this part of content in the thumbnail according tothe user's instruction.

(2) The smart desktop may detect in real time whether there is a changein the content. When detecting that there is an update in the content,the smart desktop may directly update this part of content in thethumbnail and display the updated content. This manner may also bereferred to as an instant thumbnail updating manner. This procedure maylead to a change of a block size. As shown in FIG. 6, “{circle around(6)} update prompt” illustrates a result of content update.

In the above procedure, when the content is displayed in the thumbnail,block division may be performed according to a user correlation, i.e.,blocks may be divided according to content written by users, whichrequires the smart desktop knows which content is written by which user.Therefore, it is necessary to authenticate the user of written content.

A user authentication solution according to an embodiment includes: whenthe user starts to write, the smart desktop allows the user to writefreely. At this time, the system does not know identity information ofthe user. The smart desktop receives the written content input by theuser in real time. At the same time, the smart desktop records ahandwriting characteristic of the written content such as a pressure, aspeed, an acceleration, a stroke characteristic, etc., and records awriting assistant characteristic such as position coordinates, timeinformation, a writing direction, and a text line direction for thewriting.

According to the handwriting characteristic and the writing assistantcharacteristic of the written content, a user characteristic of thecurrent user may be generated via a characteristic retrieving algorithmand is automatically saved in the smart desktop. When a user writescontinuously, the smart desktop receives content written by the user tobe authenticated, and generates a user characteristic to beauthenticated based on a handwriting characteristic and a writingassistant characteristic of the content written by the user to beauthenticated, i.e., the user characteristic will be updated in realtime.

At this time, the smart desktop matches the user characteristic to beauthenticated and the saved user characteristic, and determines theuser's identity based on various aspects such as a handwritingcharacteristic, writing position coordinates, a time continuity, awriting direction, a text line direction, a writing area, and writtencontent semantic information. If the user characteristic to beauthenticated matches the saved user characteristic, the authenticationof the user identity succeeds. Otherwise, a new account (a useridentifier) is created for the current user. The position coordinates,time information, writing direction, text line direction, and writtencontent semantic information for the writing are also saved.

After the user authentication succeeds, the content written by the usermay be saved along with the corresponding user identifier of the user.Subsequently, it is possible to know which content on the smart desktopis written by which user according to the saved user identifier and thecorresponding written content.

In some embodiments, the smart desktop may be implemented by anindependent device, or may be implemented by a plurality ofinterconnected associated devices.

If the smart desktop includes interconnected associated devices, eachuser may use an associated device (e.g., each user uses a tabletcomputer). Each user may view content written by other users on otherassociated devices by calling out a thumbnail on his associated device.The associated devices of the users may be connected via a wireless(Wi-Fi, infrared, Bluetooth, etc.) or wired manner. After the associateddevice used by the user (the viewer) calling out the thumbnail detects athumbnail calling operation of the user, the associated device obtainscontent displayed by other associated devices, detects relationshippositions of the associated devices, takes relative positions of theassociated devices as relative positions of the content on the smartdesktop, and takes content written by each user on each associateddevice as content written by each user on the same associated device atthe same time. The associated device used by the viewer displays thecontent of each associated device in the thumbnail according to therelationship positions of the associated devices.

The associated device used by the viewer may detect the relativeposition of each associated device via at least one of the followingmanners.

(1) Determining the Relative Position of Each Associated Device Based ona Network Signal Detection Result of Each Associated Device.

For example, each associated device transmits a detection result of itswireless communication signal (e.g., 2G, 3G, 4G, etc.) to the associateddevice used by the viewer. The associated device determines a positionrelationship between the associated device and other associated devicesaccording to the detection result of its wireless communication signaland a detection result transmitted by the other associated devices, soas to obtain the relative position of each associated device.

In addition, the associated device used by the viewer may also determinethe relative position of each associated device according to a combineddetection result of more or more kinds of signals between the associateddevice and the other associated devices, such as an infraredcommunication signal, a Bluetooth communication signal, or a wirelesslocal area network.

(2) Determining the Relative Position of Each Associated Device Based ona Relative Position Setting Value Provided by Each Associated Device.

When the user initially uses the smart desktop or when one or more usersrequest to call out the thumbnail, the smart desktop displays a relativeposition configuration interface on the associated device used by eachuser. The configuration interface may include a diagram for the smartdesktop consisting of the associated devices. The user may select aposition where he is currently located on the diagram of the smartdesktop. Each associated device obtains a relative position settingvalue according to a selection operation of the user. The associateddevices may exchange their relative position setting values with eachother. The associated device used by the viewer determines a positiontopology of each associated device according to the relative positionsetting values of the associated devices, so as to determine therelative position of each associated device.

When displaying the thumbnail, the thumbnail may occupy the whole screenof the viewer. The screen may be divided into several parts and eachpart displays the summarization of the content of each user. Otherfunctions are similar to those of the thumbnail on the smart desktop.

According to an embodiment, the user is able to view content written byother users in the thumbnail. In addition, the user is able to edit thecontent written by the other users. The smart desktop receives a contentediting instruction triggered by the user, and edits content in thethumbnail according to the content editing instruction, wherein theediting may include inserting, deleting, etc.

For content insertion: the user views the content written by the otherusers in the thumbnail, and selects a position for writing content via amanner such as long pressing. At this time, a cursor prompt may beprovided at this position, representing that the user wants to insertcontent at this position. The user writes the content to be inserted atthis position.

For content deletion: the user views the content written by the otherusers in the thumbnail and selects content to be deleted via a mannersuch as drawing a line on the screen. At this time, an identifier suchas a strikethrough line may be provided on the selected content,representing that the content is to be deleted.

In addition, the user may select content to be modified via a mannersuch as selecting a closed region on the touch screen. At this time, anidentifier such as a dotted rectangle may be provided for the selectedcontent, representing that the content is to be modified. The userinputs modified content. The smart desktop deletes original content andreplaces it with the modified content.

According to an embodiment, the editing to the content may besynchronized with the displaying area corresponding to the user to whichthe content belongs in real time. For example, the user A writes content“basketball is an interesting sport”. The user B modifies via thethumbnail the above content written by the user A to “basketball is aninteresting sport, and is very fit for office workers”. During anediting procedure of the user B, the smart desktop provides asynchronized editing for the content in the displaying area of the userA. The user A is able to see the editing procedure of the user B. Duringthe editing, a user identifier (a head image, etc.) of the user B may bedisplayed in the displaying area of the user A, such that the user Aknows which user is editing the content.

Also, according to an embodiment, the edited content may be in a font ofthe editing user, or a font of the user whose content is being edited.The user may also set whether to allow other users to edit his content,or allow or forbid which user to edit his content. In addition, ifanother user edits the content of the user, the user may select whetherto accept the editing.

Embodiment 2

Hereinafter, the following solution will be described in detail: adisplaying instruction for content of a smart desktop is detected; andthe content is displayed on the smart desktop based on a designateddisplaying manner. The designated displaying manner may include:displaying a summarization of the content based on a text attributeassociated with the content.

Scenario: the user A arrives during the meeting and wants to viewrecorded content of other users associated with a subject that he isinterested in.

Solution: a summarization based on a text attribute associated with thecontent, also referred to as a subject summarization.

The subject summarization may be provided according to an operation ofthe user A via at least one of the following manners:

(1) Voice Operation

The user A calls out the subject summarization via a voice manner, e.g.,saying “I want to see the subject summarization”. At this time, thesmart desktop displays a subject summarization frame in an areaincluding no content in front of the user. If the area in front of theuser is filled up, content in an area right in front of the user withthe same size as the subject summarization frame is hidden to displaythe subject summarization frame.

(2) Physical Button Operation

The user calls out the subject summarization frame by pressing a buttonon the smart desktop.

(3) Touch Screen Operation

The user calls out the subject summarization frame by touching a screen,e.g., double clicking the screen. At this time, the smart desktopdisplays the subject summarization frame on a position where the usertouches the screen. If there is content on this position, the content inan area with the same size as the subject summarization frame is hiddento display the subject summarization frame.

(4) Gesture Operation

The user calls out the subject summarization frame via a gesture action,e.g., waving a hand. At this time, the smart desktop captures thegesture action of the user via a camera and displays the subjectsummarization frame in an area with no content in front of the user. Ifthe area in front of the user is filled up, content in an area right infront of the user with the same size as the subject summarization frameis hidden to display the subject summarization frame.

(5) External Controller Operation

The user calls out the subject summarization frame by operating anexternal controller, e.g., pressing a button on a stylus pen. At thistime, the smart desktop displays the subject summarization frame on aposition pointed by the stylus pen. If there is content on thisposition, content in an area with the same size as the subjectsummarization frame is hidden to display the subject summarizationframe.

After a user operation is detected, a configuration interface may bedisplayed. The user is able to set the subject summarization in theconfiguration interface, including the following.

1. Determine Whether the Subject Summarization Includes a Summarizationof Non-Handwritten Content

If it is determined that merely a summarization of handwritten contentis displayed, merely the summarization of the handwritten content on thesmart desktop is displayed in the subject summarization, whereas asummarization of non-handwritten content such as text, an image, slides,and a video is not displayed.

If it is determined that both the summarization of the handwrittencontent and the summarization of the non-handwritten content aredisplayed, not only the summarization of the handwritten content on thesmart desktop but also the summarization of the non-handwritten contentsuch as text, an image, slides, and a video are displayed in the subjectsummarization.

If it is determined that merely the summarization of the non-handwrittencontent is displayed, merely the summarization of the non-handwrittencontent such as text, an image, slides, and a video is displayed in thesubject summarization.

2. Determine Whether Content of the User Himself is Displayed in theSubject Summarization

If it is determined to display the content of himself, the content ofthe user A (the viewer) is displayed in the subject summarization.

If it is determined not to display the content of himself, the contentof the user A (the viewer) is not displayed in the subjectsummarization.

3. Select a Size and a Position for the Subject Summarization

If a size and a position of the subject summarization are determinedautomatically, an initial size of the subject summarization frame is astandard size set in a system and a displaying position is determinedautomatically. The smart desktop preferably displays the subjectsummarization frame in a blank area in a displaying area correspondingto the user. If a size of the blank area in the corresponding displayingarea is slightly smaller than the standard size for the subjectsummarization frame set in the system (e.g., a difference between thesize of the blank area in the corresponding displaying area and thestandard size for the subject summarization frame set in the system issmaller than a preset area threshold), the size of the subjectsummarization frame is slightly reduced to fit for the blank area in thecorresponding displaying area. However, if the size of the blank area inthe corresponding displaying area is far smaller than the standard sizefor the subject summarization frame set in the system (e.g., thedifference between the size of the blank area in the correspondingdisplaying area and the standard size for the subject summarizationframe set in the system is not smaller than the preset area threshold),the user needs to proceed to the following next level options:

i. Covering Mode (Covering Existing Content)

If the user does not need to view the content written by himself whileviewing the subject summarization, this option may be selected. At thistime, the smart desktop displays the subject summarization frame in thecorresponding displaying area of the user and temporarily hides contentcovered by the subject summarization frame.

ii. Inserting Mode (Inserting into Content)

If the user needs to view the content written by himself while viewingthe subject summarization, this option may be selected. At this time,the smart desktop inserts the subject summarization frame at the end ofthe corresponding displaying area of the user. Subsequently, thehandwritten content input by the user is displayed around the subjectsummarization frame. If a displaying space is not enough for thehandwritten content of the user, a solution such as changing a layoutand dividing into multiple pages may be utilized.

If it is confirmed to manually determine the size and position of thesubject summarization frame, the user may determine the size andposition of the subject summarization frame via a touch screen gesture.

For example, the touch desktop detects that the user touches the screenon at least two positions at the same time. The smart desktop determinesthe position and size of the subject summarization frame according tothe detected touching positions of the user.

For example, when the user touches the bottom-left of the screen withthe left index finger and the top-right of the screen with the rightindex finger at the same time, the position pressed by the left indexfinger will be the bottom-left corner of the subject summarization frameand the position pressed by the right index finger will be the top-rightcorner of the subject summarization frame. At this time, if there iscontent on the position determined by the user, it is possible toproceed to the following next level options:

i. Covering Mode (Cover Existing Content); ii. Inserting Mode (Insertinto the Content)

4. Determine Whether to Display a Head Image of the User

If it is determined to display a head image of the user, a head imagecaptured by the camera in real time or a preset head image may bedisplayed in a user column in the subject summarization, wherein in thesubject summarization, the user column may be set aside of the contentof each user for displaying relevant information of the user. The viewermay differentiate summarizations of different users.

5. Determine Whether to Display a User Name

If it is determined to display a user name, the user name will bedisplayed in the user column in the subject summarization. If neitherthe user head image nor the user name is displayed, it indicates thatthe viewer does not care which user has written which content. At thistime, the summarizations of different users in each subject are notdifferentiated and are displayed in parallel. The user may not see whichcontent belongs to which user before checking detailed content of thesummarizations.

6. Select a Displaying Manner for the Subject Summarization

A displaying manner of the summarization may include a multi-levelcatalog manner. The levels may be from general to specific, e.g., in anorder of: a subject, a user, and a summarization. From the subject tothe user, it is indicated content of which user includes content aboutthe subject; and from the user to the summarization, it is indicatedwhich user has content related to which subject and is displayed in theform of a summarization. In particular, the multi-level catalogdisplaying manner may be set as follows:

(a) Horizontal Display

A left most column shows the subject, a second column shows the userwhich is identified as belonging to the subject via, e.g., tree-likeconnection, a bracket, same color, etc. A third column shows thesummarization which is identified as belonging to the user. If one userhas multiple summarizations, each one is displayed in one line in a leftalignment manner.

(b) Vertical Display

An up most row shows the subject, and a second row shows the user whichis identified as belonging to the subject via tree-like connection, abracket, same color, etc. A third row shows the summarization which isidentified as belonging to the user. If one user has multiplesummarizations, each one is displayed in new one line in a leftalignment manner.

(c) Whether to Display a Table

When the user selects to display a table, the subject summarization isrepresented in the form of a table. When the user selects not to displaya table, a table frame is hidden, and there is no table line between thesummarization and catalog.

According to an embodiment, the above configuration interface may bedisplayed after the user calls out the subject summarization. The usersets on the configuration interface. The smart desktop displays thecorresponding subject summarization according to the user's setting. Or,the user may set the displaying manner of the subject summarization inadvance. After the user calls out the subject summarization, the smartdesktop displays the corresponding subject summarization according tothe preset manner.

In addition, when the subject summarization is displayed, the user mayadjust the displaying manner of the subject summarization in real time.The smart desktop dynamically adjusts the displayed subjectsummarization. For example, the user sets in advance not to displaycontent written by himself in the subject summarization. Therefore, thesubject summarization initially presented by the smart desktop does notinclude the content written by the user himself. If the user sets todisplay the content written by himself in the subject summarization viaa preset operation (e.g., a voice, a screen gesture, etc.) when viewingthe subject summarization, the smart desktop dynamically adjusts thesubject summarization in real time to present the content written by theuser in the subject summarization.

After the setting is finished, the subject summarization is displayed inthe subject summarization frame. An implementation procedure may includethe following.

(1) Differentiating Users

When the user writes content, the smart desktop authenticates the userwriting the content according to the user authentication solutiondescribed in Embodiment 1. In particular, the smart desktop determineswhether the user writing the content is a new user according to anattribute of the user currently writing and a user attribute ofpreviously written content. If the user is a new user, a new account (auser identifier) is created in a database, and position coordinates,time information, a writing direction, a text line direction, semanticinformation of the written content, and a window opened by the user arerecorded. If the user is not a new user, the user is recognized.

(2) Retrieving a Summarization

A text summarization is retrieved for written content of each user. Thewritten content of the user may include multiple subjects. At this time,one or more summarizations may be retrieved for each subject. A lengthof the summarization does not exceed a length threshold selected by theuser during the setting.

(3) Classifying a Subject

All summarizations of all users are classified according to techniquessuch as semantic analysis. A user label is reserved for eachsummarization. For example, a summarization retrieved from content ofthe user A is “basketball has a long history”, a summarization retrievedfrom content of the user B is “basketball rules details”, and asummarization retrieved from content of a user C is “Chinese footballwill catch up the world”. At this time, the content of the user A andthe user B are classified into one subject: basketball. But the contentof the user C will not be classified into the subject.

(4) Displaying a Summarization

At this time, the subject and the corresponding summarization aredisplayed in the subject summarization frame in a form set by the user(horizontal or vertical display).

In addition, the subject summarization may have the following functions:

1. Viewing of detailed content: a content viewing instruction may bereceived from the user, and content relevant to the summarization may bedisplayed based on the content viewing instruction, such that the useris able to view detailed content of the summarization.

(a) Selection of One or Multiple Summarizations

The user may select one summarization by clicking the summarization onthe touch screen, or may select multiple summarizations by sliding overan area of the touch screen by hand.

(b) Displaying Manner for the Detailed Content

Original content corresponding to the selected summarization isdisplayed in a window automatically generated behind (if thesummarizations are horizontally displayed) or below (if thesummarizations are vertically displayed) the summarization column. Anedge length of the window is kept to be consistent with a length of arow (for horizontally displayed summarizations) or a column (forvertically displayed summarizations).

At this time, the user may select a window he wants to view and mayexpand the window by clicking on the touch screen. The window isexpanded to a viewing suitable size fitting for the subjectsummarization frame. The user may change a position and a size of thewindow via manners such as dragging.

At this time, content covered by the expanded window is hiddenautomatically. The original content corresponding to the summarizationis displayed in the window. If there is too much content, the content isautomatically divided into multiple pages for display.

A default font of the content is an original handwritten font of theuser. The viewer may change it to a printed font via a gesture. The fontsize of the content is a viewing suitable size fitting for the window.

If the user selects multiple summarizations and wants to check detailedcontent of another summarization after viewing detailed content of onesummarization, the user may roll up the window by, e.g., clicking theexpanded window on the touch screen, and then clicking anotherinterested summarization to expand its window for checking the detailedcontent.

If the user wants to simultaneously view pieces of detailed content ofseveral summarizations, the user may drag the expanded window to anadjacent position according to a requirement, instead of rolling up theexpanded window and then clicking to expand another interestedsummarization. If the expanded windows overlap each other, a laterexpanded window may overlap a previously expanded window.

(c) Searching According to a Keyword of the Summarization

After the detailed content window of the subject summarization isexpanded, the user may select a word in the summarization by, e.g.,clicking the touch screen. The word is then highlighted with a bold fontor a background highlight color. At the same time, words or phrasesrelevant to this word in the detailed content are also highlighted.

(2) Synchronous Update

When the user views the subject summarization, if another user modifiesor supplements the content, the smart desktop prompts the user on thesubject summarization when detecting that the content on the smartdesktop is updated. For example, the smart desktop may determine thatthe other user's modification is temporarily finished via a manner suchas a time sequence (e.g., the writing has been stopped for a while), andthe smart desktop displays an update prompt on the content of thesubject summarization in the form of an icon or a gradually changedbackground color. The user may provide a content updating instructionby, e.g., single clicking the touch screen on the content of the subjectsummarization. The smart desktop updates this part of content in thesubject summarization based on the user instruction.

The smart desktop may detect in real time whether there is a change inthe content. If detecting that there is a change in the content of thesmart desktop, the smart desktop may directly update this part ofcontent in the subject summarization and display the updated content.This manner is also referred to as an instant updating manner. The useris able to see an instant updating procedure of all content.

(3) Content Restriction Change

(a) Subject Broadening or Restriction

The user may broaden a subject via, e.g., a swipe to the left on thesubject, or may restrict the subject via, e.g., a swipe to the right. Ascope of the subject is determined based on a predefined structurediagram or based on an existing Wordnet technique.

For example, a summarization retrieved from content of the user A is“basketball has a long history”; a summarization retrieved from contentof the user B is “basketball rules details”; a summarization retrievedfrom content of the user C is “Chinese football will catch up with theworld”, and a summarization retrieved from content of a user D is “worldathletics championships officially ended”. At this time, the content ofthe users A and B will be classified into one subject: basketball,whereas the content of the users C and D will not be classified intothis subject. If the user regards that the subject—basketball is toosmall and wants to broaden it, the user may provide a swipe to the lefton the “basketball”. At this time, the subject changes from “basketball”to “ball games”, and the content related to football of the user C willbe classified into this subject. If the user provides a swipe to theleft again to broaden the subject, the subject may change from “ballgames” to “sports”. At this time, all of the four summarizations of theusers A, B, C, and D are classified in this subject.

(b) Summarization Length Change

If receiving a summarization length change instruction of the user, thesmart desktop changes a length of the summarization according to thesummarization length change instruction.

The user may instruct to shorten the length of the summarization via amanner such as swiping to the left on the summarization, or may instructto prolong the summarization via a manner such as swiping to the righton the summarization.

During the swiping, a ruler may be displayed to indicate a currentmaximum summarization length and an adjusted maximum summarizationlength. The maximum length of the summarization may be defined inadvance in a text summarization technique. For different maximumlengths, the content of the summarization may vary.

For example, a summarization retrieved from content of the user A is“China won the championship”. At this time, the user may swipe to theright on the summarization until the ruler shows 15 (at most 15 words).Then, the summarization will become “Chinese team won the world tabletennis championships”.

(4) Searching a Subject

(a) Calling Out a Searching Box

The user may call out a searching box in the subject summarization frameby, e.g., double clicking any position in the subject summarizationframe.

(b) Inputting a Subject

The user inputs a required subject in the searching box through, e.g., ahandwriting input and a keyboard input, etc.

After the user confirms to search, the smart desktop detects a searchinginstruction including the subject input by the user and performssearching in the summarizations based on the searching instruction.

(c) Displaying a Subject Summarization

i. Hiding Original Content

If the searched subject includes too much content and the subjectsummarization frame does not have an enough blank area, original contentin the subject summarization frame may be hidden to display the searchedsubject and the corresponding summarization.

ii. Subsequent Displaying

If the searched subject includes less content and the subjectsummarization frame has an enough blank area, the searched subject andthe corresponding summarization may be displayed following the existingsubject.

(5) Privacy Setting

The user may set a part of his content as private content via mannerssuch as selecting a closed region by a gesture and hope that suchprivate content is invisible for other users. In addition, the user mayselect to which users that he does not want to expose the privatecontent. At this time, when the smart desktop detects that the selecteduser opens the subject summarization, the private content of the user isnot presented in the summarization, i.e., the smart desktop hides theprivate content of the user in the subject summarization.

The hiding of the private information in the subject summarizationincludes: not displaying the private information, or replacing theprivate information by a private identifier.

In addition, according to an embodiment, the smart desktop mayautomatically detect whether the content of the user is private content,i.e., if content such as figures in a personal account, a user name, ora password is detected, it is determined that the corresponding contentmay be private content.

FIG. 7 is a schematic diagram illustrating a display of the subjectsummarization according to Embodiment 2 of the present disclosure.

As shown in FIG. 7, the left side shows an actual screen display of thesmart desktop, wherein a user Kim writes content about a table tennis, auser Frank writes content about a basketball, a guitar, and a piano, auser Lia writes content about a piano, and a user Mako writes contentabout a table tennis.

When the user Mako views a subject summarization (the right part in FIG.7), the system automatically displays by the subjects: sports and music,and the content written by different users are differentiated inrespective subject. For example, in the subject of sports, pieces ofcontent are arranged according to: the table tennis written by Kim, thebasketball written by Frank, and the table tennis written by Mako. Thepieces of content are represented by a summarization. The user mayselect the summarization to view detailed content. Since there may bevarious manners to determine the subject, if the user does not find aninteresting subject, he may input an interesting subject (e.g., abasketball as shown in FIG. 7). Content relevant to this subject may bepresented by users.

In some embodiments, the smart desktop may be implemented by anindependent device, or may be implemented by a plurality of associateddevices having a connection relationship.

If the smart desktop includes associated devices having a connectionrelationship, each user may use an associated device (e.g., each useruses a tablet computer). Each user may view content written by otherusers on other associated devices by calling out a subject summarizationon his associated device. The associated devices of the users may beconnected via a wireless (Wi-Fi, infrared, Bluetooth, etc.) or wiredmanner. After detecting an operation for calling out the subjectsummarization, the associated device used by the viewer obtains contentdisplayed on other associated devices. After obtaining the contentdisplayed on the other associated devices, the associated device of theviewer may bring up and display the subject summarization via the abovedescribed manner for bringing up the subject summarization by the smartdesktop.

When the subject summarization is displayed, the subject summarizationmay occupy the whole screen of the associated device of the viewer.Other functions are similar to those of the subject summarization on thesmart desktop.

Embodiment 3

Hereinafter, the following solution will be described in detail: adisplaying instruction for content of a smart desktop is detected; andthe content is displayed on the smart desktop based on a designateddisplaying manner. The designated displaying manner may include:displaying a summarization of the content based on a user attributeassociated with the content.

Scenario: the user A is far away from the user B and the user A wants toexplain his handwritten content to the user B.

Solution: a summarization based on a user attribute associated with thecontent, also referred to as a user summarization.

The user summarization may be called out according to an operation ofthe user A via at least one of the following manners:

(1) Voice Operation

The user A calls out the user summarization via a voice manner, e.g.,saying “I want to see the user summarization”. At this time, the smartdesktop displays a user summarization frame in an area including nocontent in front of the user. If the area in front of the user is filledup, content in an area right in front of the user with the same size asthe user summarization frame is hidden to display the user summarizationframe.

(2) Physical Button Operation

The user calls out the user summarization frame by pressing a button onthe smart desktop.

(3) Touch Screen Operation

The user calls out the user summarization frame by touching a screen,e.g., double clicking the screen. At this time, the smart desktopdisplays the user summarization frame on a position where the usertouches the screen. If there is content on this position, the content inan area with the same size as the user summarization frame is hidden todisplay the user summarization frame.

(4) Gesture Operation

The user calls out the user summarization frame via a gesture action,e.g., waving a hand. At this time, the smart desktop captures thegesture action of the user via a camera and displays the usersummarization frame in an area with no content in front of the user. Ifthe area in front of the user is filled up, content in an area right infront of the user with the same size as the user summarization frame ishidden to display the user summarization frame.

(5) External Controller Operation

The user calls out the user summarization frame by operating an externalcontroller, e.g., pressing a button on a stylus pen. At this time, thesmart desktop displays the user summarization frame on a positionpointed by the stylus pen. If there is content on this position, contentin an area with the same size as the user summarization frame is hiddento display the user summarization frame.

After a user operation is detected, a configuration interface may bedisplayed. The user is able to set the user summarization in theconfiguration interface, including the following.

1. Determine Whether the User Summarization Includes a Summarization ofNon-Handwritten Content

If it is determined that merely a summarization of handwritten contentis displayed, merely the summarization of the handwritten content on thesmart desktop is displayed in the user summarization, whereas asummarization of non-handwritten content such as text, an image, slides,and a video is not displayed.

If it is determined that both the summarization of the handwrittencontent and the summarization of the non-handwritten content aredisplayed, not only the summarization of the handwritten content on thesmart desktop but also the summarization of the non-handwritten contentsuch as text, an image, slides, and a video are displayed in the usersummarization.

If it is determined that merely the summarization of the non-handwrittencontent is displayed, merely the summarization of the non-handwrittencontent such as text, an image, slides, and a video is displayed in theuser summarization.

2. Determine Whether Content of the User Himself is Displayed in theUser Summarization

If it is determined to display the content of himself, the content ofthe user A (the viewer) is displayed in the user summarization.

If it is determined not to display the content of himself, the contentof the user A (the viewer) is not displayed in the user summarization.

3. Select a Size and a Position for the User Summarization

If a size and a position of the user summarization are determinedautomatically, an initial size of the user summarization frame is astandard size set in a system and a displaying position is determinedautomatically. The smart desktop preferably displays the usersummarization frame in a blank area in a displaying area correspondingto the user. If a size of the blank area in the corresponding displayingarea is slightly smaller than the standard size for the usersummarization frame set in the system (e.g., a difference between thesize of the blank area in the corresponding displaying area and thestandard size for the user summarization frame set in the system issmaller than a preset area threshold), the size of the usersummarization frame is slightly reduced to fit for the blank area in thecorresponding displaying area. However, if the size of the blank area inthe corresponding displaying area is far smaller than the standard sizefor the user summarization frame set in the system (e.g., the differencebetween the size of the blank area in the corresponding displaying areaand the standard size for the user summarization frame set in the systemis not smaller than the preset area threshold), the user needs toproceed to the following next level options:

i. Covering Mode (Covering Existing Content)

If the user does not need to view the content written by himself whileviewing the user summarization, this option may be selected. At thistime, the smart desktop displays the user summarization frame in thecorresponding displaying area of the user and temporarily hides contentcovered by the user summarization frame.

ii. Inserting Mode (Inserting into Content)

If the user needs to view the content written by himself while viewingthe user summarization, this option may be selected. At this time, thesmart desktop inserts the user summarization frame at the end of thecorresponding displaying area of the user. Subsequently, the handwrittencontent input by the user is displayed around the user summarizationframe. If a displaying space is not enough for the handwritten contentof the user, a solution such as changing a layout and dividing intomultiple pages may be utilized.

If it is confirmed to manually determine the size and position of theuser summarization frame, the user may determine the size and positionof the user summarization frame via a touch screen gesture. For example,the touch desktop detects that the user touches the screen on at leasttwo positions at the same time. The smart desktop determines theposition and size of the subject summarization frame according to thedetected touching positions of the user. For example, when the usertouches the bottom-left of the screen with the left index finger and thetop-right of the screen with the right index finger at the same time,the position pressed by the left index finger will be the bottom-leftcorner of the user summarization frame and the position pressed by theright index finger will be the top-right corner of the usersummarization frame. At this time, if there is content on the positiondetermined by the user, it is possible to proceed to the following nextlevel options:

i. Covering Mode (Cover Existing Content); ii. Inserting Mode (Insertinto the Content)

4. Determine Whether to Display a Head Image of the User

If it is determined to display a head image of the user, a head imagecaptured by the camera in real time or a preset head image may bedisplayed in a user column in the user summarization, wherein in theuser summarization, the user column may be set aside of the content ofeach user for displaying relevant information of the user. The viewermay differentiate summarizations of different users.

5. Select a Displaying Manner for the Subject Summarization

A displaying manner of the summarization may include a multi-levelcatalog manner. The levels may be from general to specific, e.g., in anorder of: a user, a subject, and a summarization. From the user to thesubject, it is indicated content of which user includes content aboutwhich subject; and from the subject to the summarization, it indicatesthe summarization of the content of the user under the subject. Inparticular, the multi-level catalog displaying manner may be set asfollows:

(a) Horizontal Display

A left most column shows the user, and a second column shows the subjectwhich is identified as belonging to the user via, e.g., tree-likeconnection, a bracket, same color, etc. A third column shows one or moresummarizations belonging to the subject.

(b) Vertical Display

An up most row shows the user, and a second row shows the subject whichis identified as belonging to the user via, e.g., tree-like connection,a bracket, same color, etc. A third row shows one or more summarizationsbelonging to the subject.

(c) Whether to Display a Table

When the user selects to display a table, the user summarization isrepresented in the form of a table. When the user selects not to displaya table, the table frame is hidden, and there is no table line betweenthe summarization and catalog.

According to an embodiment, the above configuration interface may bedisplayed after the user calls out the user summarization. The user setson the configuration interface. The smart desktop displays thecorresponding user summarization according to the user's setting. Or,the user may set the displaying manner of the user summarization inadvance. After the user calls out the user summarization, the smartdesktop displays the corresponding user summarization according to thepreset manner.

In addition, when the user summarization is displayed, the user mayadjust the displaying manner of the user summarization in real time. Thesmart desktop dynamically adjusts the displayed user summarization. Forexample, the user sets in advance not to display content written byhimself in the user summarization. Therefore, the user summarizationinitially presented by the smart desktop does not include the contentwritten by the user himself. If the user sets to display the contentwritten by himself in the user summarization via a preset operation(e.g., a voice, a screen gesture, etc.) when viewing the usersummarization, the smart desktop dynamically adjusts the usersummarization in real time to present the content written by the user inthe user summarization.

After the setting is finished, the user summarization is displayed inthe user summarization frame. An implementation procedure may includethe following.

(1) Differentiating Users

When the user writes content, the smart desktop authenticates the userwriting the content according to the user authentication solutiondescribed in Embodiment 1, which is not repeated herein.

(2) Retrieving a Summarization

A text summarization is retrieved for written content of each user. Thewritten content of the user may include multiple subjects. At this time,one or more summarizations may be retrieved for each subject. A lengthof the summarization does not exceed a length threshold selected by theuser during the setting.

(3) Classifying a Subject

All summarizations of all users are classified according to techniquessuch as semantic analysis.

(4) Displaying a Summarization

At this time, the subject and the corresponding summarization aredisplayed in the subject summarization frame in a form set by the user(horizontal or vertical display).

In addition, the subject summarization may have the following functions:

1. Viewing of detailed content: a content viewing instruction may bereceived from the user, and content relevant to the summarization may bedisplayed based on the content viewing instruction, such that the useris able to view detailed content of the summarization.

(a) Selection of One or Multiple Summarizations

The user may select one summarization by clicking the summarization onthe touch screen, or may select multiple summarizations by sliding overan area of the touch screen by hand.

(b) Displaying Manner for the Detailed Content

Original content corresponding to the selected summarization isdisplayed in a window automatically generated behind (if thesummarizations are horizontally displayed) or below (if thesummarizations are vertically displayed) the summarization column. Anedge length of the window is kept to be consistent with a length of arow (for horizontally displayed summarizations) or a column (forvertically displayed summarizations). At this time, the user may selecta window he wants to view and may expand the window by clicking on thetouch screen. The window is expanded to a viewing suitable size fittingfor the subject summarization frame. The user may change a position anda size of the window via manners such as dragging. At this time, contentcovered by the expanded window is hidden automatically. The originalcontent corresponding to the summarization is displayed in the window.If there is too much content, the content is automatically divided intomultiple pages for display. A default font of the content is an originalhandwritten font of the user. The viewer may change it to a printed fontvia a gesture. The font size of the content is a viewing suitable sizefitting for the window. If the user selects multiple summarizations andwants to check detailed content of another summarization after viewingdetailed content of one summarization, the user may roll up the windowby, e.g., clicking the expanded window on the touch screen, and thenclicking another interested summarization to expand its window forchecking the detailed content. If the user wants to simultaneously viewpieces of detailed content of several summarizations, the user may dragthe expanded window to an adjacent position according to a requirement,instead of rolling up the expanded window and then clicking to expandanother interested summarization. If the expanded windows overlap eachother, a later expanded window may overlap a previously expanded window.

(c) Searching According to a Keyword of the Summarization

After the detailed content window of the subject summarization isexpanded, the user may select a word in the summarization by, e.g.,clicking the touch screen. The word is then highlighted with a bold fontor a background highlight color. At the same time, words or phrasesrelevant to this word in the detailed content are also highlighted.

(2) Synchronous Update

When the user views the subject summarization, if another user modifiesor supplements the content, the smart desktop prompts the user on thesubject summarization when detecting that the content on the smartdesktop is updated. For example, the smart desktop may determine thatthe other user's modification is temporarily finished via a manner suchas a time sequence (e.g., the writing has been stopped for a while), andthe smart desktop displays an update prompt on the content of thesubject summarization in the form of an icon or a gradually changedbackground color. The user may provide a content updating instructionby, e.g., single clicking the touch screen on the content of the subjectsummarization. The smart desktop updates this part of content in thesubject summarization based on the user instruction.

The smart desktop may detect in real time whether there is a change inthe content. If detecting that there is a change in the content of thesmart desktop, the smart desktop may directly update this part ofcontent in the subject summarization and display the updated content.This manner is also referred to as an instant updating manner. The useris able to see an instant updating procedure of all content.

(3) Content Restriction Change

(a) Subject Broadening or Restriction

The user may broaden a subject via, e.g., a swipe to the left on thesubject, or may restrict the subject via, e.g., a swipe to the right. Ascope of the subject is determined based on a predefined structurediagram or based on an existing Wordnet technique. For example, asummarization retrieved from content of the user A is “basketball has along history”; a summarization retrieved from content of the user B is“basketball rules details”; a summarization retrieved from content ofthe user C is “Chinese football will catch up with the world”, and asummarization retrieved from content of the user D is “world athleticschampionships officially ended”. At this time, the content of the usersA and B will be classified into one subject: basketball, whereas thecontent of the users C and D will not be classified into this subject.If the user regards that the subject-basketball is too small and wantsto broaden it, the user may provide a swipe to the left on the“basketball”. At this time, the subject changes from “basketball” to“ball games”, and the content related to football of the user C will beclassified into this subject. If the user provides a swipe to the leftagain to broaden the subject, the subject may change from “ball games”to “sports”. At this time, all of the four summarizations of the usersA, B, C, and D are classified in this subject.

(b) Summarization Length Change

If receiving a summarization length change instruction of the user, thesmart desktop changes a length of the summarization according to thesummarization length change instruction.

The user may instruct to shorten the length of the summarization via amanner such as swiping to the left on the summarization, or may instructto prolong the summarization via a manner such as swiping to the righton the summarization. During the swiping, a ruler may be displayed toindicate a current maximum summarization length and an adjusted maximumsummarization length. The maximum length of the summarization may bedefined in advance in a text summarization technique. For differentmaximum lengths, the content of the summarization may vary. For example,a summarization retrieved from content of the user A is “China won thechampionship”. At this time, the user may swipe to the right on thesummarization until the ruler shows 15 (at most 15 words). Then, thesummarization will become “Chinese team won the world table tennischampionships”.

(4) Searching a Subject

(a) Calling Out a Searching Box

The user may call out a searching box in the subject summarization framevia, e.g., double clicking any position in the subject summarizationframe.

(b) Inputting a Subject

The user inputs a required subject in the searching box through, e.g., ahandwriting input and a keyboard input, etc.

After the user confirms to search, the smart desktop detects a searchinginstruction including the subject input by the user and performssearching in the summarizations based on the searching instruction.

(c) Displaying a Subject Summarization

i. Hiding Original Content

If the searched subject includes too much content and the subjectsummarization frame does not have an enough blank area, original contentin the subject summarization frame may be hidden to display the searchedsubject and the corresponding summarization.

ii. Subsequent Displaying

If the searched subject includes less content and the subjectsummarization frame has an enough blank area, the searched subject andthe corresponding summarization may be displayed following the existingsubject.

(5) Privacy Setting

The user may set a part of his content as private content via mannerssuch as selecting a closed region by a gesture and hope that suchprivate content is invisible for other users. In addition, the user mayselect to which users that he does not want to expose the privatecontent. At this time, when the smart desktop detects that the selecteduser opens the subject summarization, the private content of the user isnot presented in the summarization, i.e., the smart desktop hides theprivate content of the user in the subject summarization.

The hiding of the private information in the subject summarizationincludes: not displaying the private information, or replacing theprivate information by a private identifier.

In addition, according to an embodiment, the smart desktop mayautomatically detect whether the content of the user is private content,i.e., if content such as figures in a personal account, a user name, ora password is detected, it is determined that the corresponding contentmay be private content.

FIG. 8 is a schematic diagram illustrating a display of the subjectsummarization according to Embodiment 2 of the present disclosure.

As shown in FIG. 8, the left side shows an actual screen display of thesmart desktop. A user may write content on different positions due to achange of his position. As shown in FIG. 8, the user Kim writes twoparts of content, wherein one is about a basketball and is on one sideof the screen (an initial position of him) and the other is about atable tennis and is on a current position of him. Pieces of contentwritten by different users may be differentiated by different colors. Atthis time, if the user Mako wants to view content written by a certainuser, he may call out a user summarization (the right part in FIG. 8).In the user summarization, all pieces of content written by users areshown in the form of a subject summarization. For example, if the userMako wants to view content written by Kim, he may clearly see in theuser summarization that Kim has written content in subjects: basketballand table tennis. At this time, he may select one subject and viewdetailed content.

In some embodiments, the smart desktop may be implemented by anindependent device, or may be implemented by a plurality of associateddevices having a connection relationship.

If the smart desktop includes associated devices having a connectionrelationship, each user may use an associated device (e.g., each useruses a tablet computer). Each user may view content written by otherusers on other associated devices by calling out the subjectsummarization on his associated device. The associated devices of theusers may be connected via a wireless (Wi-Fi, infrared, Bluetooth, etc.)or wired manner. After detecting an operation for calling out thesubject summarization, the associated device used by the viewer obtainscontent displayed on other associated devices. After obtaining thecontent displayed on the other associated devices, the associated deviceof the viewer may bring up and display the user summarization via theabove described manner for bringing up the user summarization by thesmart desktop.

When the user summarization is displayed, the user summarization mayoccupy the whole screen of the associated device of the viewer. Otherfunctions are similar to those of the user summarization on the smartdesktop.

According to an embodiment, the user is able to view content written byother users on a summarization (user summarization or subjectsummarization). In addition, it is also possible to edit the contentwritten by the other users. The smart desktop receives a content editinginstruction triggered by the user, and edits content in a thumbnailbased on the content editing instruction, wherein the editing mayinclude inserting, deleting, etc.

For content insertion: the user views the content written by the otherusers in the summarization, and selects a position for writing contentvia a manner such as long pressing. At this time, a cursor prompt may beprovided at this position, representing that the user wants to insertcontent at this position. The user writes the content to be inserted atthis position.

For content deletion: the user views the content written by the otherusers in the summarization and selects content to be deleted via amanner such as drawing a line on the screen. At this time, an identifiersuch as a strikethrough line may be provided on the selected content,representing that the content is to be deleted.

In addition, the user may select content to be modified via a mannersuch as selecting a closed region on the touch screen. At this time, anidentifier such as a dotted rectangle may be provided for the selectedcontent, representing that the content is to be modified. The userinputs modified content. The smart desktop deletes original content andreplaces it with the modified content.

According to an embodiment, the editing to the content may besynchronized with a displaying area corresponding to the user to whichthe content belongs in real time. For example, the user A writes content“basketball is an interesting sport”. The user B modifies via thethumbnail the above content written by the user A to “basketball is aninteresting sport, and is very fit for office workers”. During anediting procedure of the user B, the smart desktop provides asynchronized editing for the content in the displaying area of the userA. The user A is able to see the editing procedure of the user B. Duringthe editing, a user identifier (a head image, etc.) of the user B may bedisplayed in the displaying area of the user A, such that the user Aknows which user is editing the content.

Also, according to an embodiment, the edited content may be in a font ofthe editing user, or a font of the user whose content is being edited.The user may also set whether to allow other users to edit his content,or allow or forbid which user to edit his content. In addition, ifanother user edits the content of the user, the user may select whetherto accept the editing.

Embodiment 4

When multiple users write by using a smart desktop, since each user sitsat a different position and has a different direction, it isinconvenient for the user to share his written content with other usersor write on the position in front of other users.

In view of this, according to an embodiment, a displaying instructionfor content of the smart desktop is detected, and the content isdisplayed on the smart desktop based on a designated displaying manner.The content includes content to be shared. The designated displayingmanner includes: determining a user receiving the content to be shared;and displaying the content on the smart desktop based on the determineduser.

Scenario: the user A wants to share content written by him with the userB.

Solution: sharing with a designated user.

A user (sharing user) who shares content with others may designate auser (receiving user) receiving the content to be shared in advance andthen designate the content to be shared, or may designate the content tobe shared and then designate the user receiving the content to beshared.

Designation of the user: according to an embodiment, if detecting auser's particular operation (e.g., pressing a sharing button, etc.), forconfirming sharing, the smart desktop displays user identifiers (headimages, user names, etc.) of other users in a current displaying area.The user may select a user receiving content to be shared according tothe user identifiers. For example, the user may select a user by longpressing a real time head image or a name of the user. At this time, thehead image or name of the user is highlighted, indicating that it hasbeen selected and the content is to be shared with the user. Inaddition, according to an embodiment, a user with which the content isto be shared may be designated based on a user summarization or athumbnail indicating a relative position.

Designation of content: the user may designate existing content as thecontent to be shared, or may be designate content written in real timeas the content to be shared. The user opens an input box and inputs thecontent to be shared.

(a) Designating Existing Content

The sharing user may select the content to be shared in his existingcontent by, e.g., selecting a closed region on the touch screen. If theuser selects to share the content with a particular receiving user, thecontent is automatically inserted following current displayed content ofthe receiving user. A font size may be consistent with that of thecurrent displayed content of the receiving user. A font type may beconverted into that of the sharing user or the receiving user viatechniques such as handwriting recognition and handwriting combination.

Optionally, a layout of the shared content may be automatically arrangedaccording to a displaying area corresponding to the receiving user onthe smart desktop. For example, if there is much content to be sharedwhich may not be completely displayed in a blank area of thecorresponding displaying area of the receiving user on the smartdesktop, the font type of the content may be adjusted or the content maybe displayed in multiple pages.

(b) Designating Real Time Written Content

The sharing user calls out a writing box by, e.g., long pressing aposition in the corresponding displaying area of the smart desktop. Asize of the writing box may be set by a system in advance or set by theuser, or may be automatically adjusted according to the current blankarea. During a writing procedure of the sharing user, the size of thewriting box may be automatically expanded and original content on thescreen covered by the writing box may be hidden. At this time, thereceiving user synchronously displays the writing procedure of thesharing user in the corresponding displaying area of the smart desktop.A font size may be consistent with that of currently displayed contentof the receiving user. A font type may be the same as that of thesharing user or the receiving user.

The smart desktop may automatically arrange a layout of the content tobe shared according to the displaying area of the receiving user. Forexample, if there is too much content to be shared which may not becompletely displayed in the blank area of the corresponding displayingarea of the receiving user on the smart desktop, the font type of thecontent may be adjusted or the content may be displayed in multiplepages.

The sharing user may indicate that he has finished writing by, e.g.,double clicking the writing box. At this time, the sharing user isprompted to save the content in the writing box. If the sharing userselects to save the content, the content in the writing box isautomatically added to the blank area in the corresponding displayingarea of the sharing user on the smart desktop or added to the end of theexisting content. Then, the writing box of the sharing user is closed.At the same time, a prompt is displayed in the corresponding displayingarea of the receiving user on the smart desktop, prompting the receivinguser to select whether to accept the added or modified content. If thereceiving user accepts, the sharing content is automatically added tothe original content or modifications are made to the original content,and the layout is rearranged. If the content added after the layoutrearrangement may not be displayed due to an insufficient blank area,multiple pages may be provided for the content, or a layout of otherblocks may be adjusted to generate a blank area for displaying the addedcontent.

According to an embodiment, the smart desktop adjusts a displayingdirection of the content to be shared according to a viewing directionof the user receiving the shared content (receiving user).

FIG. 9 is a schematic diagram illustrating sharing with a designateduser according to Embodiment 4 of the present disclosure.

As shown in FIG. 9, the left part illustrates an actual screen displayof the smart desktop. The user Kim writes content about a table tennis,the user Frank writes content about a basketball, a guitar, and a piano,the user Lia writes content about a piano, and the user Mako writescontent about a table tennis. When the user Kim checks a subjectsummarization (as shown in the right part of FIG. 9), the systemautomatically displays by the subjects: sports and music, and pieces ofcontent written by different users are differentiated in respectivesubject. For example, in the subject of sports, the pieces of contentare displayed in an order of: the table tennis written by Kim, thebasketball written by Frank, and the table tennis written by Mako,wherein the content is represented by a summarization. The user maycheck detailed content by selecting the summarization. The user Kim mayselect the content about the table tennis as the content to be sharedand select the user Frank as a user with which the content is to beshared, so as to share the content about the table tennis of Kim withthe user Frank.

In addition, according to an embodiment, the sharing user may designatemultiple users receiving the shared content (receiving users). Thus, theshared content will be displayed in displaying areas of the designatedmultiple receiving users simultaneously.

Embodiment 5

According to an embodiment, a displaying instruction for content on asmart desktop is detected; and the content is displayed on the smartdesktop based on a designated displaying manner. The content includescontent to be shared. The designated displaying manner includes:determining a displaying position of the content to be shared, anddisplaying the content on the smart desktop based on the determineddisplaying position.

Scenario: the user A wants to share his content with the users B and C,and the users B and C are sitting on adjacent seats.

Solution: sharing based on designated positions.

A user sharing content (sharing user) may firstly designate a displayingposition for displaying the content to be shared and then designate thecontent to be shared, or may firstly designate the content to be sharedand then designate the displaying position for displaying the content tobe shared. According to an embodiment, the displaying position fordisplaying the content to be shared may be designated via a thumbnailindicating a relative position.

Designation of a position: the sharing user may select a blank position(e.g., a position in the middle of two content blocks) in the thumbnailindicating the relative position by clicking on a touch screen,indicating that the user wants to share content at this position.

Designation of content: the sharing user may designate existing contentas the content to be shared, or designate content written in real timeas the content to be shared. For example, the user may open a writingbox and input the content to be shared. A method of designating contentis similar to that described with reference to FIG. 4 and is notrepeated herein.

According to an embodiment, the smart desktop may detect a position of areceiving user via a camera and adjust the displaying position of thecontent to be shared to a central position of a viewing angle of one ormore users.

FIG. 10 is a schematic diagram illustrating sharing based on adesignated position according to Embodiment 5 of the present disclosure.

As shown in FIG. 10, the left part illustrates an actual screen displayof the smart desktop. The user Kim writes content about a table tennis,the user Frank writes content about a basketball, the user Lia writescontent about a piano, and the user Mako writes content about a pianoand a guitar. The user Lia selects a blank area close to the user Frankand the user Kim in a thumbnail indicating a relevant position byclicking the touch screen (indicating that the user Lia wants to sharecontent on this blank area). The smart desktop shares the content sharedby Lia (e.g., content written by Lia in real time) in the blank area.Therefore, the users Frank and Kim may conveniently view the sharedcontent.

In addition, according to an embodiment, the sharing user may designatemultiple displaying positions for displaying the content to be shared.As such, the content to be shared is displayed on the multipledisplaying positions designated by the sharing user.

Embodiment 6

According to an embodiment, when sharing content, a sharing region maybe set on a smart desktop. After receiving a content sharing determininginstruction, the smart desktop copies content to be shared to thesharing region in response to the content sharing determininginstruction. Other users may view the shared content conveniently in thesharing region.

Hereinafter, the following solution is described in detail: a displayinginstruction for content of the smart desktop is detected; and thecontent is displayed on the smart desktop based on a designateddisplaying manner. The content includes content to be shared. Thedesignated displaying manner includes: setting a sharing region on thesmart desktop, and copying the content to be shared to the sharingregion in response to a content sharing determining instruction.

Scenario: multiple users are discussing a problem and need to share,view and obtain their respective content.

Solution: sharing region

An operation for triggering to display a sharing region is set inadvance. For example, if palms of users are placed on a screen, itindicates that the users want to open the sharing region. At this time,the smart desktop displays the sharing region on the screen whendetecting the above operation of the users. A position, a size, and ashape of the sharing region are described as follows.

(1) Position and Shape

The center of the sharing region may be the center of the whole screen.The shape may be set by the user, may be relevant to a screen shape ofthe smart desktop (e.g., a round sharing region for a round smartdesktop), or may be set to be elliptical in default.

(2) Size

The size of the sharing region may be preset by a system. The edge ofthe sharing region is reachable for all users. At this time, if somepieces of displayed content are covered by the sharing region, blocks ofthe pieces of displayed content are automatically adjusted to a blankarea of the corresponding user on the smart desktop. If the blank areaof the user is not big enough, the content may be put in multiple pagesto reduce the size, so as to place the content in the blank area. If theuser has no blank area in the displaying area on the smart desktop, thecontent is moved to the end of another block of the user and isdisplayed in another page. The user may view different pages in the areathrough a screen slide.

A color of the sharing region may be different from other areas on thesmart desktop, so as to be conveniently recognized by users.

About Sharing of the Content

FIG. 11 is a schematic diagram illustrating a sharing region accordingto Embodiment 6 of the present disclosure. As shown in (a) of FIG. 11, auser sharing content (sharing user or sharer) selects content to beshared (e.g., by selecting a closed region), or the user sharing thecontent (the sharing user or the sharer) writes the content to be sharedin real time. As shown in (b) of FIG. 11, the sharing user (the sharer)drags the content to be shared to a sharing region via an operation(such as an upward swipe), indicating that the content is to bedisplayed in the sharing region. The smart desktop copies the content tobe shared to the sharing region, wherein a direction of the content tobe shared is aligned with the edge of the sharing region. The sharinguser (sharer) may hide the content to be shard to a designated user. Atthis time, if the content to be shared is switched to the front of theuser for whom the content is set to be hidden, the content to be sharedis automatically hidden.

About Viewing of the Shared Content:

As shown in (c) of FIG. 11, the sharing region may rotate by its center.When the user's hand touches the edge of the sharing region, the sharingregion enters into a rotating mode. A plate rotates with the sliding ofthe user's hand and a rotating speed is consistent with a sliding speed.At this time, only this user may rotate the sharing region. When thesharing user (sharer) rotates the sharing region, the content in thesharing region rotates with the rotation of the sharing region and keepsits original relative position. At this time, the user is able to searchfor his desired content. If the hand of the sharing user (sharer) hasleft the sharing region for a certain period of time, the plate entersinto a static state again.

About obtaining of the shared content:

As shown in (d) of FIG. 11, a user expecting to obtain the sharedcontent (viewer) may rotate the sharing region through manners such asdragging on the touch screen, obtain some content in the sharing region,and display the content in his displaying area.

(1) The user expects to obtain the shared content whose displayingposition is not close. At this time, the user expecting to obtain theshared content may initiatively rotate the sharing region, so as to beclose to the shared content. In response to a sharing region rotationinstruction, the smart desktop rotates the sharing region.

(2) When the shared content rotates to nearby the user expecting toobtain the shared content, the rotation of the sharing region may bestopped by a sharing region rotation stop instruction triggered by theuser.

(3) When the shared content is stopped nearby the user expecting toobtain the shared content, the user may copy the shared content from thesharing region to his displaying area, so as to obtain the contentshared by another user.

About Anonymous Sharing:

After putting the shared content into the sharing region, the sharinguser may display the shared content and a user identifier (a head image,a signature, etc.) corresponding to the content in the sharing region.In addition, according to an embodiment, the user may also performanonymous sharing. The user may set via a preset operation (e.g., clickan anonymous sharing button) so that, after the shared content is putinto the sharing region, merely the shared content is displayed in thesharing region and the corresponding user identifier is not displayed.In addition, font conversion may be provided for the content to beshared, e.g., a handwriting font of the sharing user may be convertedinto a printed type.

Embodiment 7

When a user writes text or a formula or draw a picture by using a smartdesktop, a writing space may be insufficient for the user since thereare pieces of existing content on the smart desktop.

In view of this, according to an embodiment, existing content which hasa position conflict with an input position of content currently beingwritten is detected, a displaying instruction for the existing contentis triggered when detecting the existing content, and the existingcontent is temporarily hidden.

Hereinafter, the following solution is described in detail: a displayinginstruction for content of the smart desktop is detected; and thecontent is displayed on the smart desktop based on a designateddisplaying manner. The content includes: existing content which has aposition conflict with an input position of content currently beingwritten. The detecting of the displaying instruction for the content ofthe smart desktop includes: detecting the existing content which has theposition conflict with the input position of the content currently beingwritten, and triggering a displaying instruction for the existingcontent when detecting the existing content. The designated displayingmanner includes: temporarily hiding the existing content.

Scenario: content A exists on the smart desktop. A user inputs content Bon the left side of the existing content A. When inputting the contentB, an inputting space is insufficient for the content B due to theexistence of the content A.

FIG. 12 is a schematic diagram illustrating a writing conflict accordingto Embodiment 7 of the present disclosure.

In order to overcome a writing conflict, the present disclosure includesthe following:

(a) Implementing a Function Corresponding to a Writing Conflict

1. Implemented with the start of the smart desktop; 2. implemented by auser via an option configuration provided on a touch screen; 3.Implemented by the user via a voice instruction; 4. Implemented by theuser via a handwritten instruction; 5. implemented automatically whenthere is an input operation; and 6. implemented by the user via anexternal device such as a stylus pen.

(b) Detecting a Writing Conflict and Hiding Conflicting Content in RealTime.

The present disclosure provides a position detection hiding mode and aconflict prediction hiding mode. The user may set via manners such as avoice instruction, a handwritten instruction, a desktop screen touch,and an external device. The position detection hiding mode and theconflict prediction hiding mode are respectively described as follows.

Position detection hiding mode: when the user inputs text or a formulaor draws a picture, a system detects whether there is a conflict betweena position of current writing of the user (i.e., an input position ofcontent currently input by the user) and existing content on thedesktop. If there is a conflict, in order to enable continuous writingof the current writing, locally hiding is provided for the existingcontent having the conflict on the position of the current writing.Thus, the user is able to keep on inputting.

Conflict prediction hiding mode: when the user inputs text or a formulaor draws a picture, the system predicts a space required by the user forthe current input according to following content recently input by theuser: a handwriting characteristic, text content and structure (acharacter, a word, a sentence and a paragraph), formula content andformat, and a picture shape and size, and detects whether there is aconflict between the required space and a position of the existingcontent, i.e., predicting whether the existing content has a positionconflict on the input position with the content currently being written.If there is a conflict, the system provides locally hiding for theexisting content having the conflict according to the predicted requiredspace on the predicted position where the conflict may exist, so thatthe user may keep on inputting.

FIG. 13 is a schematic diagram illustrating a first embodiment of adisplay after writing conflict processing according to Embodiment 7 ofthe present disclosure. As shown in FIG. 13, a writing conflict may besolved by hiding some existing pieces of content.

(c) Displaying Adjustment of the Conflicting Content after Input

When detecting that the input of the content currently being written isfinished, the smart desktop may determine a display resuming mannerbased on a semantic correlation between the existing content and thecontent currently being written.

The display resuming manner includes: resuming a display after moving aposition of the existing content and/or the content currently beingwritten, i.e., a whole content moving mode; or resuming a display afterlayout rearrangement of the existing content and/or the contentcurrently being written, i.e., a local content layout rearrangementmode.

The user may set via manners such as a voice instruction, a handwritteninstruction, a desktop screen touch, and an external controller. Thewhole content moving mode and the local content layout rearrangementmode are described as follows:

Whole Content Moving Mode:

FIG. 14 is a schematic diagram illustrating a second embodiment of adisplay after writing conflict processing according to Embodiment 7 ofthe present disclosure. As shown in FIG. 14, the system detects in realtime whether there is a position conflict with existing content duringan input of new content. After the input of the new content is finished,if the system detects that there is a conflict during the input, thesystem respectively determines complete areas for the new content andthe existing content based on a handwriting characteristic, text contentand structure (a character, a word, a sentence, and a paragraph),formula content and format, a picture shape and size, an input time,etc. After the complete areas of them are determined, in order to removethe conflict and resume a display of the existing content that ishidden, the system provides an adjustment for relative positions of thecontent of the two complete areas. The existing content having theconflict may be moved to resume the display of the existing content. Or,the new content having the conflict may be moved to resume the displayof the existing content. Or, both the new content and the existingcontent having the conflict may be moved, so as to resume the display ofthe existing content.

Local Content Layout Rearrangement Mode:

FIG. 15 is a schematic diagram illustrating a third embodiment of adisplay after writing conflict processing according to Embodiment 7 ofthe present disclosure. As shown in FIG. 15, the system detects in realtime during writing new content whether the new content has a positionconflict with existing content and records a conflict position. After aninput of the new content is finished, in order to remove the conflictand resume a display of the conflicting content that is hidden, thesystem performs a local layout rearrangement for the newly input contentand/or the existing content on the conflict position. A layout of theexisting content having the conflict may be rearranged on the conflictposition, the display of the existing content may be resumed and alayout of the newly input content may be rearranged, or the layouts ofboth the newly input content and the existing content may be rearranged.

A layout of a complete picture, a formula, and text may be rearrangedaccording to their priorities, so as to ensure that a shape of thepicture is not changed and a continuity of the formula is not affectedduring the rearrangement. It is preferable to rearrange the layout ofthe text, then the formula, and last the picture. For example, if thenewly input content is a picture and the existing content having theconflict is text content, it may be preferable to rearrange a layout ofthe text content, so as to ensure that a shape of the picture is notchanged.

In addition, according to an embodiment, after the input is finished,the display of the content having the conflict may be adjusted:

1) if the content currently written by the user is relevant to theconflicting existing content, the display may be adjusted according to asurrounding displaying manner as shown in FIG. 15;

2) if the content currently written by the user is irrelevant to theconflicting existing content, the display may be adjusted according to asurrounding displaying manner as shown in FIG. 14;

3) if the content currently being written and the conflicting existingcontent are written by the same user, the display may be adjustedaccording to the surrounding displaying manner as shown in FIG. 14; and

4) if the content currently being written and the conflicting existingcontent are not written by the same user, the display may be adjustedaccording to the surrounding displaying manner as shown in FIG. 15.

(d) Limiting the Writing Conflict Relevant Function:

1. Limited with the close of the smart desktop; 2. Limited by the uservia a provided touch screen option configuration; 3. limited by the uservia a voice instruction; 4. Limited by the user via a handwritteninstruction; 5. automatically limited when there is no operation; and 6.Limited by the user via an external associated device such as a styluspen.

Embodiment 8

During the usage of a smart desktop, an external object such as a cup, abag, a cellphone, a notebook, an elbow, etc. may cover characters orother content on a screen, which is inconvenient for a user.

In view of the above, according to an embodiment, an occluding object onthe smart desktop is detected; if content of the smart desktop iscovered by the occluding object, a displaying instruction for thecovered content is triggered; and a layout of the covered content isrearranged.

Hereinafter, the following solution is described in detail: a displayinginstruction for content of the smart desktop is detected; and thecontent on the smart desktop is displayed based on a designateddisplaying manner. The content includes: content being covered by anoccluding object on the smart desktop. The detecting of the displayinginstruction for the content of the smart desktop includes: detecting theoccluding object on the smart desktop, and when the content of the smartdesktop is covered by the occluding object, triggering the displayinginstruction for the content. The designated displaying manner includes:rearranging a layout of the covered content.

Scenario: the user A puts a cup on the smart desktop. Content displayedon the smart desktop is covered by the cup. FIG. 16 is a schematicdiagram illustrating Embodiment 8 of the present disclosure.

In order to overcome the occlusion, the present disclosure includes thefollowing:

1. Automatic Occlusion Adjusting Mode

The user starts an automatic occlusion adjusting mode via, e.g., a) avoice instruction; the user A may emit a voice instruction such as a“start the automatic occlusion adjusting mode”. At this time, theautomatic occlusion adjusting mode is switched on; b) a physical buttonoperation: the user may switch on the automatic occlusion adjusting modeby pressing a particular button on the smart desktop; c) a touch screenoperation: the user may switch on the automatic occlusion adjusting modeby touching the screen, e.g., triple clicking on the screen; d) agesture operation: the user provides a gesture action, e.g., waving hishand three times. At this time, a camera may capture the gesture actionof the user and the automatic occlusion adjusting mode may be switchedon; e) an operation using an external controller: the user may operatean external controller, e.g., pressing a button on a stylus pen, toswitch on the automatic occlusion adjusting mode; and f) a handwritingoperation: the user may input a fixed mode instruction via handwriting,e.g., writing “switch on the automatic occlusion adjusting mode” toswitch on the automatic occlusion adjusting mode.

After entering into the automatic occlusion adjusting mode, occlusionprocessing includes the following.

(a) Layout analysis and content recognition are provided for the contentpresented on the screen via a sensor, detection and recognition areprovided for the occluding object on the desktop, and an occluded plane,occluded content, and a section (e.g., a paragraph) where the occludedcontent is located are determined.

A result of the layout analysis includes, but is not limited to: i. acharacter and a word; ii. a text line; iii. a paragraph; iv. textcontent with coherent meaning; v. an image; and vi. text and an imagewith coherent meaning.

The detection and recognition of the occluding object includes, but isnot limited to: i. a type and a shape of the occluding object; ii. adegree of transparency of the occluding object; and iii. a shape of theoccluded plane.

In some embodiments, a layout of an occluded object is rearranged,including: rearranging a layout of the occluded content based on atleast one of: a character continuity of the occluded content, a textline continuity of the occluded content, a paragraph continuity of theoccluded content, a semantic continuity of the occluded content, animage continuity of the occluded content, and a correlation between textand an image of the occluded object. In particular, detailed content tobe adjusted is determined first, and then a layout of the content to beadjusted is rearranged.

(b) According to the above result, the detailed content to be adjustedis determined based on at least one of the following:

i. a character continuity of the occluded content, ii. a text linecontinuity of the occluded content, iii. a paragraph continuity of theoccluded content, iv. a semantic continuity of the occluded content, v.an image continuity of the occluded content, and vi. a correlationbetween the text and the image of the occluded object.

(c) After optimal adjusting content is determined, the layout of theoccluded content is rearranged according to the following manners: i.determining a size of an area available for redisplaying the occludedcontent; ii. rearranging a layout of the optimal adjusting contentdetermined according to the available area and the occluded content; andiii. rearranging content rearranged around the occluded plane.

After entering into the automatic occlusion adjusting mode, theocclusion processing may be adjusted in real time according to a changein the occluding object and the content displayed on the desktop. FIG.17 is a schematic diagram illustrating a first embodiment of a displayafter the occlusion processing according to Embodiment 8 of the presentdisclosure. A result after the automatic occlusion adjustment is asshown in FIG. 17.

According to an embodiment, before the rearrangement of the layout ofthe occluded content, the smart desktop may firstly detect a focus pointof the user. If an occluded position does not overlap the focus point ofthe user, the rearrangement of the layout of the occluded content maynot be performed at this time. If the occluded position overlaps thefocus point of the user, the layout of the occluded content may berearranged. The focus point of the user may be detected via anassociated device such as a camera and a visual focus may be taken asthe focus point of the user. A position currently edited by the user mayalso be taken as the focus point of the user.

According to an embodiment, when the rearrangement of the layout of theoccluded content is performed, the smart desktop may present text in theform of a summarization, so as to save an available displaying space.

The automatic occlusion adjusting mode may be switched off via at leastone of the following: a) a voice operation: the user A emits a voicesuch as “end the automatic occlusion adjusting mode” to switch off theautomatic occlusion adjusting mode; b) a physical button operation: theuser may switch off the automatic occlusion adjusting mode by pressing abutton on the smart desktop; c) a touch screen operation: the userswitches off the automatic occlusion adjusting mode by touching thescreen, e.g., clicking the screen four times continuously; d) a gestureoperation: the user provides a gesture action, e.g., waving a hand fourtimes. At this time, the camera may capture the gesture action of theuser and the automatic occlusion adjusting mode may be switched off; e)an operation using an external controller: the user operates theexternal controller, e.g., pressing a button on the stylus pen, to endthe automatic occlusion adjusting mode; f) a handwriting operation: theuser writes a fixed mode instruction, e.g., writing “end the automaticocclusion adjusting mode” to switch off the automatic occlusionadjusting mode; g) close of the smart desktop: the mode is switched offdue to the close of the smart desktop, e.g., the meeting is over; and h)change of the current user: if the user A switches on the automaticocclusion adjusting mode on a position a of the smart desktop and movesto a position b. Later, a new user C joins the meeting and moves to theposition a. A system detects that the current user on the position a ischanged and switches off the automatic occlusion adjusting mode for theuser C. At the same time, the system keeps the automatic adjusting modefor the user A on the position b.

2. Manual Occlusion Adjusting Mode

The user may switch on a manual occlusion adjusting mode via thefollowing: a) a voice instruction: the user A may emit a voice, e.g.,“switch on the manual occlusion adjusting mode”. At this time, themanual occlusion adjusting mode is switched on; b) a physical buttonoperation: the user switches on the manual occlusion adjusting mode bypressing a button on the smart desktop; c) a touch screen operation: theuser switches on the manual occlusion adjusting mode by touching thescreen, e.g., double clicking the screen three times; d) a gestureoperation: the user provides a gesture action, e.g., waving a hand fivetimes, at this time, the camera may capture the gesture action of theuser and the manual occlusion adjusting mode is switched on; e) anoperation using an external controller: the user operates the externalcontroller, e.g., pressing a button on the stylus pen, to switch on themanual occlusion adjusting mode; and f) a handwriting operation: theuser writes a fixed mode instruction, e.g., writing “switch on themanual occlusion adjusting mode” to switch on the manual occlusionadjusting mode.

After entering the manual occlusion adjusting mode, the occlusionprocessing includes the following.

(a) Layout analysis and content recognition are provided for the contentpresented on the screen via a sensor, detection and recognition areprovided for the occluding object on the desktop, and the occludedplane, the occluded content, and the section (e.g., a paragraph) wherethe occluded content is located are determined.

A result of the layout analysis includes, but is not limited to: i. acharacter and a word; ii. a text line; iii. a paragraph; iv. textcontent with coherent meaning; v. an image; and vi. text and an imagewith coherent meaning.

The detection and recognition of the occluding object includes, but isnot limited to: i. a type and a shape of the occluding object; ii. adegree of transparency of the occluding object; and iii. a shape of theoccluded plane.

In some embodiments, a layout of an occluded object is rearranged,including: rearranging a layout of the occluded content based on atleast one of: a character continuity of the occluded content, a textline continuity of the occluded content, a paragraph continuity of theoccluded content, a semantic continuity of the occluded content, animage continuity of the occluded content, and a correlation between textand an image of the occluded object. In particular, detailed content tobe adjusted is determined first, and then a layout of the content to beadjusted is rearranged.

(b) According to the above result, the detailed content to be adjustedis determined based on at least one of the following: i. a charactercontinuity of the occluded content, ii. a text line continuity of theoccluded content, iii. a paragraph continuity of the occluded content,iv. a semantic correlation of the occluded content, v. an imagecontinuity of the occluded content, and vi. a correlation between thetext and the image of the occluded object.

(c) After optimal adjusting content is determined, the layout of theoccluded content is rearranged according to the following manners: i.the user drags content (text or an image) around the occluded content ina certain direction via an operation on the touch screen; ii. Theoptimal adjusting content determined by a system is moved according to adragging operation of the user, and a moving speed and distance may bedefined according to a dragging strength, speed, and distance on thetouch screen; iii. If the optimal adjusting content may not becompletely displayed after the movement due to the restriction of adisplaying area, non-occluded content in the optimal adjusting contentafter the movement may be hidden or a font type may be adjusted toensure that the user is able to see the occluded content preferably.

FIG. 18 is a schematic diagram illustrating a second embodiment of adisplay after the occlusion processing according to Embodiment 8 of thepresent disclosure. A result of the manual occlusion adjustment is asshown in FIG. 18.

In addition, when the layout of the occluded content is rearranged, thesmart desktop may present text in the form of a summarization, so as tosave an available displaying area as much as possible. Subsequently,detailed content corresponding to the summarization may be presentedaccording to the user's operation.

The manual occlusion adjusting mode may be switched off via at least oneof the following: a) a voice operation: the user A emits a voice such as“end the manual occlusion adjusting mode” to switch off the manualocclusion adjusting mode; b) a physical button operation: the user mayswitch off the manual occlusion adjusting mode by pressing a button onthe smart desktop; c) a touch screen operation: the user switches offthe manual occlusion adjusting mode by touching the screen, e.g.,clicking the screen four times continuously; d) a gesture operation: theuser provides a gesture action, e.g., waving a hand six times. At thistime, the camera may capture the gesture action of the user and themanual occlusion adjusting mode may be switched off; e) an operationusing an external controller: the user operates the external controller,e.g., pressing a button on the stylus pen, to end the manual occlusionadjusting mode; f) a handwriting operation: the user writes a fixed modeinstruction, e.g., writing “end the manual occlusion adjusting mode” toswitch off the manual occlusion adjusting mode; g) close of the smartdesktop: the mode is switched off due to the close of the smart desktop,e.g., the meeting is over; and h) change of the current user: if theuser A switches on the manual occlusion adjusting mode on the position aof the smart desktop and moves to the position b. Later, a new user Cjoins the meeting and moves to the position a. The system detects thatthe current user on the position a is changed and switches off themanual occlusion adjusting mode for the user C. At the same time, thesystem keeps the manual adjusting mode for the user A on the position b.

In addition, the user may view the occluded content by calling out athumbnail, wherein the calling of the thumbnail is described withreference to Embodiment 1 and is not repeated herein.

In view of the above, the present disclosure provides a solution for theoccluded content, which is able to ensure a normal display of theoccluded content.

Embodiment 9

When multiple users use a smart desktop, the multiple users may operatethe same control simultaneously. But a control resource may be limited.Therefore, there may be a resource conflict.

1. Take a Recording Resource as an Example

Generally, the smart desktop may have only one microphone. At this time,if the multiple users need to record a voice, there may be a conflict.

The solution of the present disclosure is as follows: assigning avirtual recording button for each user, and the virtual recording buttonof the user may be called out according to an operation of the user,e.g., a) a voice operation; the user emits a voice, e.g., saying“recording”. At this time, the virtual recording button may emerge in ablank area in front of the user. If the area in front of the user isfilled up, content in an area with the same size as the virtualrecording button in front of the user may be hidden to display thevirtual recording button; b) a physical button operation: the user callsout the virtual recording button via a button on the smart desktop; c) atouch screen operation: the user touches a screen, e.g., long pressingthe screen. At this time, the virtual recording button will be displayedon a position of the screen where the user touches the screen. If thereis content on this position, the content in an area with the same sizeas the virtual recording button may be temporarily hidden to display thevirtual recording button; d) a gesture operation: the user provides agesture action, e.g., waving a hand. At this time, a camera may capturethe user's gesture action and the virtual recording button may emerge ina blank area in front of the user. If the area in front of the user isfilled up, content in an area with the same size as the virtualrecording button may be temporarily hidden to display the virtualrecording button; and e) an operation using an external controller: theuser operates an external controller, e.g., pressing a button of astylus pen. At this time, the virtual recording button is displayed on aposition pointed by the stylus pen. If there is content on thisposition, the content in an area with the same size as the virtualrecording button may be temporarily hidden to display the virtualrecording button.

FIG. 19 is a schematic diagram illustrating recording resource conflictprocessing according to Embodiment 9 of the present disclosure.

A user may bring up a virtual recording button to start recording whenhearing content that he is interested in, and may stop the recording bypressing the virtual recording button again. First, when detecting avirtual recording button pressing event, the smart desktop receives theuser's recording start instruction and starts the recording, i.e.,starting a recording apparatus of the user to record a voice. Whendetecting that all users select to stop recording, the smart desktopreceives a recording stop instruction of each user and stops therecording, i.e. stopping a recording apparatus of each user. As shown inFIG. 19, the user A presses a virtual recording button at a time t1. Thesmart desktop starts a recording apparatus to record a voice. The usersB and C respectively press virtual recording buttons at times t2 and t3.At a time t4, the user B presses the virtual recording button again tostop recording. At a time t5, the user A presses the virtual recordingbutton again to stop the recording. At a time t6, the user C presses thevirtual recording button again to stop the recording. The user D pressesa virtual recording button at a time t7 and presses the virtualrecording button again at a time t8 to stop the recording. The times t1to t8 are in a time order. It is known from the above that an actualrecording period of the smart desktop is t1-t6 and t7-t8. If the timest6 and t7 are the same, the actual recording period of the smart desktopis t1-t8.

When saving a recording file, the smart desktop generates the recordingfile for each user according to a recording start instruction and arecording stop instruction of the user. In particular, according to therecording start instruction and recording stop instruction of each user,the smart desktop may obtain a starting time and a stopping time of eachuser using a recording function. According to the starting time and thestopping time of each user using the recording function, the smartdesktop searches an audio file of actual recording for the recordingfile corresponding to the user.

When any user wants to play recording, he may bring up a recordingcontent playing interface via a manner such as long pressing the screen.A progress bar on the recording content playing interface starts from anearliest time and ends at a latest time from among all users. Recordingcontent is divided into several key points according to a change incontent based on voice recognition and semantic analysis. According toan embodiment, the recording content playing interface includes anidentifier of each important period and/or content obtained by voicerecognition of each important period. The important period is a periodcorresponding to a key point determined based on a period where the userperforms a recording operation and/or the change in the content.Therefore, in the recording content playing interface, the identifier isprovided at a starting time and an ending time of each key point, and isalso provided at a starting time and an ending time of each user'srecording. The user may select to play at any key point, or may selectto play recording of a particular user. The user may also select to saveaudio content.

For example, as shown in FIG. 19, the user D records merely a key point5 of the meeting, but the user D wants to view content of other keypoints. Audio content recorded by the users A, B, and C includes contentof key points 1 to 4. Therefore, the user D may bring up a playinginterface as shown in FIG. 19, and selects to play content of other keypoints or content recorded by other users in the playing interface.

According to an embodiment, when recorded content is played, it ispossible to select to play a sound. In addition, in a multi-user meetingscenario, the playing of the sound may affect other users. Therefore, itis also possible to convert an audio into text content via techniquessuch as voice and semantic recognition and display the text content inan interface.

In view of the above, an embodiment realizes parallelized invoking of arecording resource of the smart desktop for different users, and thus aresource utilization rate is increased.

2. Take a Time Reminding Resource as an Example

An alarm clock (time reminder) function may be provided by an audioplaying resource of a device. When multiple users use the smart desktop,the multiple users may have set their alarm clocks (time reminders).Therefore, there may be a resource conflict from among the multipleusers.

According to an embodiment, a displaying instruction for content of thesmart desktop is detected; and the content is displayed on the smartdesktop based on a designated displaying manner.

The content includes time reminding information.

The detecting of the displaying instruction for the content of the smartdesktop includes the following: detecting a reminding time set by auser, and if the reminding time set by the user arrives, triggering thedisplaying instruction for the content.

The designated displaying manner includes: if users of the smart desktopset the same reminding time, when the reminding time arrives, playingthe time reminding information via at least one of: a voice, an icon,and text; or, if the users using the smart desktop do not set the samereminding time, when the reminding time arrives, playing the timereminding information via at least one of: an icon and text.

FIGS. 20A and 20B are schematic diagrams illustrating time remindingresource conflict processing according to Embodiment 9 of the presentdisclosure. FIG. 20A shows time reminding resource conflict processingfor the same alarm time according to Embodiment 9. FIG. 20B shows timereminding resource conflict processing for different alarm timesaccording to Embodiment 9.

As shown in FIG. 20A, if multiple users set the same alarm time, it isregarded that all users need to be reminded at this time. Therefore, analarm sound is played when the alarm time arrives.

In addition, if each user has different content to be reminded or oneuser has multiple pieces of content to be reminded at one time point, itis possible to display an icon reminder and/or a text reminder in adisplaying area of each user when the alarm time arrives. It is alsopossible to both play the alarm sound and display the icon reminderand/or the text reminder.

In addition, according to an embodiment, content written by a user maybe correlated with a recording file. If it is detected that the contentwritten by the user is correlated with content in the recording file, arecording playing button is added following the content written by theuser. The user may select to play recording or display correspondingrecording content. Thus, the user is able to refer to the recordingcontent when writing.

As shown in FIG. 20B, if multiple users do not set the same alarm time,it is regarded that a particular user needs to be reminded at this time.If an alarm sound is played, other users may be affected. Therefore,when an alarm time of each user arrives, an icon reminder and/or a textreminder may be displayed in a displaying area of the user.

During the above processing, when the icon reminder and/or the textreminder is displayed in the displaying area of the user, a focus pointof the user may be determined firstly and then the icon reminder and/orthe text reminder may be displayed at the focus point of the user. Avisual focus of the user may be detected by a device such as a cameraand the visual focus of the user may be taken as the focus point of theuser. Or, a position currently edited by the user may be taken as thefocus point of the user. During the display of the icon reminder and/orthe text reminder, if the focus point of the user moves, a position ofthe icon reminder and/or the text reminder moves accordingly.

In view of the above, an embodiment realizes parallelized invoking of atime reminding resource of the smart desktop for different users, andthus a resource utilization rate is increased.

In view of the above, according to an embodiment, a solution formultiple user invoking requests for the same resource of the smartdesktop is provided, which is able to solve a conflict of the multipleuser invoking requests and ensure normal and reasonable invoking of aresource of the smart desktop.

Embodiment 10

When multiple users use a smart desktop, different users may writedifferent content. In addition, the multiple users may write the samecontent at the same time.

According to an embodiment, the smart desktop may determine promptinformation for content currently being written according to existingwritten content and display the determined prompt information.

In particular, 1) during a writing procedure of a user, the smartdesktop provides a writing prompt according to content written by otherusers. 2) Content written in real time by the user may be an emailaddress, a date, a time, a personal name, a formula, etc. At this time,the user may be prompted about correctness of content to be written orcontent having been written. 3) The content written in real time by theuser may be text. At this time, a summarization for content not havingbeen written by the user may be retrieved from content having beenwritten by other users, and the summarization may be provided to thecurrent user. The current user may check detailed information of thesummarization.

Hereinafter, detailed processing manners are provided.

1. Keyword Prompt

Prompt information for content currently being written is determinedaccording to a keyword of the content currently being written.

In Particular:

A keyword is retrieved from the content currently being written, and acontext keyword corresponding to the keyword retrieved from the contentcurrently being written is obtained according to the keyword and acontext keyword retrieved from existing written content. The promptinformation for the content currently being written is determinedaccording to the obtained context keyword.

Alternatively, a context keyword is retrieved from the content currentlybeing written, a keyword corresponding to the context keyword retrievedfrom the content currently being written is obtained according to thekeyword and a context keyword retrieved from the existing writtencontent, and prompt information is determined according to the obtainedkeyword.

(a) Define a Keyword

Herein, a keyword includes, but is not limited to, a grammatical propernoun. In general, it may be a key noun including detailed and specialmeaning. For example,

1. Name (e.g., a Personal Name, an Address, a Country Name, anOrganization Name, a Chemical Material Name, Etc.)

The following are some examples: “Tom” in “Tom is a smart kid”,“tom233@gmail.com” in “his email address tom233@gmail.com”, “China” in“China will host the 2022 winter Olympics”, and “acetaminophen” in “coldmedicines usually contain acetaminophen”.

2. Date (e.g., a Year, a Month, a Date, a Festival, a Week, Etc.)

The following are some examples: “December 15” in “the birthday of HAN,Meimei is December 15”, “Wednesday” in “we will hold a college meetingon Wednesday”, and “national day” in “the firm will be closed for 7 daysfor the national day”.

3. Digit (and a Corresponding Quantifier)

The following are some examples: “3.1415926” in “Pi is 3.1415926”,“13527194204” in “the telephone number of ZHANG San is 13527194204”,“5324” in “the total travel expense to be paid by LI Si is 5324dollars”, and “7” in “the firm will be closed for 7 days for thenational day”.

4. General Noun Keyword (May Include an Adherent Adjunct)

The following are some examples: “elevator card” in “statistics ofelevator cards of all employees are to be collected today”, and “freshfruit, mineral water, notebook” in “items need to be bought for themeeting: fresh fruit, mineral water, notebook”.

5. Key Verb or Phrase

The following is an example: “call the leader” in “tell ZHANG Santomorrow to call the leader”.

(b) Establish a Correspondence Table Between a Keyword and a ContextKeyword

For content written by a user, a system finds a keyword of the contentwritten by the user via manners such as semantic analysis and apredefined keyword table, detects a keyword in context, establishes acorrespondence with a noun, and saves the correspondence in acorrespondence table. For content written by all users, a correspondencetable is adjusted. The adjustment of the correspondence table isperformed in real time during a writing procedure of all users.

For Example:

His email address is tom233@gmail.com; we will hold a college meeting onWednesday; the total travel expense to be paid by LI Si is 5324 dollars;items to be bought for the meeting: fruits, mineral water, notebook.

Thus, a correspondence table may be as shown in Table 1:

TABLE 1 Keyword 2 (context Correspondence Keyword 1 keyword) 1 His emailaddress tom233@gmail.com 2 Wednesday College meeting 3 Travel expense tobe paid 5324 dollars by LI Si 4 Items need to be bought for fruits,mineral water, the meeting notebook

(c) Determine a Similarity and Calculate Reliability

For content written by other users, if a detected proper noun alreadyexists in a correspondence table or has a high similarity to a word inthe correspondence table, processing may be performed according to thefollowing cases:

i. The detected proper noun is in the correspondence table, and acorresponding context keyword corresponds to that in the correspondencetable.

In this case, it indicates that a new user has written the same content.At this time, the reliability of the correspondence may be increasedaccording to an algorithm.

ii. The detected proper noun has a high similarity to a proper noun inthe correspondence table but they are not the same.

For example, content written by the user A includes a sentence “ZHANGSan will meet LI Si”. At this time, “ZHANG San” and “meet LI Si” will beadded to a correspondence table. Then, the user B writes a sentence“ZHANG Er will meet LI Si”. At this time, “ZHANG Er” has a highsimilarity to the “ZHANG San” in the correspondence table but the systemmay not determine whether one user has a clerical error (one of the twowords is wrong), or both of them are right. Therefore, “ZHANG Er” and“meet LI Si” are added in the correspondence table as a new item, and itis indicated that it is similar to “ZHANG San” and “meet LI Si”. If oneof several similar correspondences has a similarity higher than athreshold (i.e. multiple users have written the same content), thesystem may regard that this correspondence is correct and others may beincorrect.

iii. The detected proper noun is in the correspondence table but acorresponding context keyword does not correspond to that in thecorrespondence table.

One proper noun may correspond to multiple keywords. For example,content written by the user A includes a sentence: ZHANG San will meetLI Si on Sunday. At this time, the correspondence of keywords may besaved in a table as shown in Table 2. Then, the user B writes asentence: ZHANG San will meet LI Si. After detecting the keyword “ZHANGSan”, the system inquires the correspondence table and finds thatkeywords corresponding to “ZHANG San” include not only “meet LI Si” butalso “Sunday”. At this time, the system may increase the reliability ofthe correspondence between “ZHANG San” and “meet LI Si” according to analgorithm.

If the system detects that a context keyword is not completelyconsistent with a corresponding keyword in a correspondence table, thesystem may add a new keyword item to the correspondence table.

If the system detects that a context keyword is more than or partiallysimilar to a corresponding keyword in a correspondence table, the systemmay add an extra keyword to the correspondence table, and increase thereliability of the correspondence of an existing keyword according to analgorithm.

TABLE 2 correspondence keyword 1 keyword 2 keyword 3 1 ZHANG San Sundaymeet LI Si

(d) Detect a Keyword Currently Written by a User and Inquire aCorrespondence Table

When a user is writing a keyword (may be a keyword or a context keywordin a correspondence table), the system finds the keyword or a keywordhaving a high similarity to this keyword from the correspondence table.

(e) Provide a Prompt for a User, Including:

i. Correction

A keyword is retrieved from content currently being written, and if asimilarity between this keyword and a keyword retrieved from existingwritten content is higher than a threshold, prompt information for thecontent currently being written is determined according to the keywordretrieved from the existing written content.

In particular, when a keyword written by a user has a high similaritybut is not completely the same as a keyword in a correspondence table,the system may prompt the user of the keyword with the high reliabilityexcept the keyword written by the user in the form of, e.g., charactersin a light color. If there are several different similar keywords, aprompt may be provided in the form of a list. If the user selects akeyword by, e.g., clicking a touch screen, the originally writtenkeyword is automatically corrected to be the prompted keyword and isdisplayed with the same font and size.

FIG. 21 is a schematic diagram illustrating a first embodiment ofproviding a prompt according to Embodiment 10 of the present disclosure.In FIG. 21, users Hyemin Park and Fulong Wang have written an emailaddress of the user Fulong Wang. When the smart desktop detects that anemail address currently written by the user Lina is incorrect, in orderto prompt the user Lina that she may have a clerical error, the smartdesktop provides a prompt for the user Lina, wherein the prompt mayinclude the correct email address. The user Lina clicks on the touchscreen to acknowledge the clerical error. The smart desktop corrects theemail address originally written by Lina to the correct email address.

ii. Prompt

A keyword is retrieved from content currently being written, a contextkeyword corresponding to the keyword retrieved from the contentcurrently being written is determined according to the keyword and acontext keyword retrieved from existing written content, and promptinformation for the content currently being written is determined basedon the determined context keyword.

In particular, a keyword written by a user is the same as a keyword in acorrespondence table. If the keyword has only one corresponding contextkeyword, the corresponding context keyword is displayed in a light colorfollowing content written by the user to provide a prompt. If thekeyword has multiple corresponding context keywords, writing of the useris continuously detected. If it is detected that the user writes contentrelevant to the corresponding context keywords, the context keywords aredisplayed in a light color following content written by the user toprovide a prompt.

Alternatively, a keyword written by a user is the same as a contextkeyword in a correspondence table. If the context keyword has only onecorresponding keyword, the corresponding context keyword is displayed ina light color following content written by the user to provide a prompt.If the context keyword has multiple corresponding keywords, writing ofthe user is continuously detected. If it is detected that the userwrites content relevant to the corresponding keywords, the contextkeyword is displayed in a light color following content written by theuser to provide a prompt.

FIG. 22 is a schematic diagram illustrating a second embodiment ofproviding a prompt according to Embodiment 10 of the present disclosure.In FIG. 22, the smart desktop detects that the user Fulong Wang haswritten his email address. The smart desktop saves the email address anda user identifier of the user Fulong Wang correspondingly. When thesmart desktop detects that the user Hyemin Park is writing “Fulong Wang”and writing “Email”, the smart desktop predicts that the user HyeminPark will write the email address of Fulong Wang. Therefore, the smartdesktop provides a prompt for the user Hyemin Park using a light color.The prompt may include the saved email address of Fulong Wang, so as toprompt the user Hyemin Park to omit writing the detailed email address.The user Hyemin Park clicks the touch screen to acknowledge the prompt.The smart desktop automatically adds the saved email address of FulongWang behind the previously written keyword “Email”.

According to an embodiment, when providing an intelligent prompt, it ispossible to determine whether the intelligent prompt is required andwhat prompt is to be provided according to a correspondence table. Forexample, content currently written by a user may be compared withcontent having been written by other users in real time. It may bedetermined whether to provide a prompt and what prompt is to be providedaccording to a compared result.

2. Summarization Prompt

A summarization for existing written content is retrieved from theexisting written content. A summarization for content currently beingwritten is retrieved from the content currently being written. Thesummarization for the existing written content and the summarization forthe content currently being written are compared, and prompt informationfor the content currently being written is provided based on a comparedresult.

In particular:

(a) Retrieval of a summarization: instant summarization retrieval isperformed on content having been written by all users. Multiplesummarizations may be retrieved.

(b) Summarization prompt, including:

i. Automatic Prompt

When a user writes content, the user may trigger a summarization promptif he has no idea. For example, the user leaves a blank area on aposition where the summarization prompt needs to be displayed. Thesystem retrieves the content currently written by the user and retrievesa summarization according to a trigger operation of the user. The systemcompares the retrieved summarization with a summarization of contenthaving been written by all users and determines the summarization promptfor the content currently written by the user. When detecting the blankarea left by the user, the system may automatically display thesummarization of content which is not written by the user but has beenwritten by other users in the blank area left by the user in a lightcolor. If the user has written something before leaving the blank area,the system may select to display a summarization relevant to thatwritten by the user.

In addition, the user may write a special symbol to trigger thesummarization prompt when having no idea, e.g., ellipsis points. Afterdetecting the special symbol, the system may automatically provide aprompt. If there are several prompts to be provided, they may bedisplayed via a drop-down menu to save a space.

When the user selects a prompted summarization via a manner such asclicking on the touch screen, detailed content corresponding to thesummarization may be expanded. The user may see the detailed content andselect required content through selection or dragging. At this time, theselected content may be inserted into the blank area left by the user inan appropriate font size and the user's font type.

ii. Manual Prompt

The user may trigger to display the summarization by a voice, a button,a gesture, etc. For example, the user may trigger to display thesummarization by his palm's sliding on the screen. At this time, thesummarization of the content which is not written by the user but hasbeen written by other users may be presented in the blank area of theuser in a light color. FIG. 23 is a schematic diagram illustrating athird embodiment of providing a prompt according to Embodiment 10 of thepresent disclosure.

In FIG. 23, the user Hyemin Park has written detailed content of threeparagraphs. The smart desktop generates a summarization for each of thethree paragraphs. The user Lina writes only content of two paragraphs,wherein the second paragraph is blank. The smart desktop presents thesummarization of the second paragraph written by Hyemin Park to Lina asa prompt. The user Fulong Wang does not write any content. The smartdesktop presents the summarizations of the three paragraphs written byHyemin Park to Fulong Wang as a prompt.

In view of the above, according to an embodiment, prompt information maybe displayed when a user is writing. Thus, a prompt for content which isnot written by the user but has been written by other users or asummarization of such content may be provided intelligently. Throughanalyzing the content written by all users, the writing experience ofthe user is improved, the efficiency of cooperation of multiple users isimproved, and repeated writing by the multiple users is avoided.

Embodiment 11

Content Exchanged with Multiple Associated Devices:

An embodiment may be also applied to a scenario using multipleassociated devices. A smart desktop establishes a connectionrelationship with the associated devices, and exchanges content of thesmart desktop and/or content of the associated devices with theassociated devices based on the connection relationship. For example,the smart desktop may be implemented as a smart whiteboard in teaching.Hereinafter, the smart whiteboard is taken as an example for describingan embodiment.

FIG. 24 is a schematic diagram illustrating a smart whiteboard accordingto Embodiment 11 of the present disclosure.

Application scenario 1: a teacher uses a smart whiteboard and studentsuse terminals (e.g., pads). The smart desktop is the whiteboard and theterminals used by the students are associated devices or referred to asassociated terminals.

1. Starting Up and Connection.

After the smart whiteboard and the associated terminals are started up,a connection between them may be established via the following manners:

(1) Bluetooth: the smart whiteboard or/and the associated terminalsswitch on a Bluetooth function and may find a surrounding whiteboardor/and associated terminals which have switched on the Bluetoothfunction. A data connection between them may be implemented via aconnection request or an automatic connection. (2) Infrared: the smartwhiteboard or/and the associated terminals switch on an infraredfunction and may find a surrounding whiteboard or/and associatedterminals which have switched on the infrared function. A dataconnection between them may be implemented via a connection request oran automatic connection. (3) Wireless network: the smart whiteboardor/and the associated terminals may implement a data connection by beingconnected to the same local network.

The above communication manners including Bluetooth, infrared, andwireless network are examples for establishing the connection betweenthe smart whiteboard and the associated terminals. One of ordinary skillin the art would know that the description is merely some examples andis not used for restricting the protection scope of the presentdisclosure.

2. Identity Authentication

After the connection between the smart whiteboard and the associatedterminals is established, identity authentication and privilege settingare performed. An identity characteristic of each user and a setprivilege may be shared between the smart whiteboard and the associatedterminals via the communication manners such as Bluetooth, infrared, andwireless network. It should be noted that: since the identityauthentication is not directly related to the number of the smartwhiteboard or the associated terminals, the number of users may belarger than the number of actual hardware smart whiteboard andassociated terminals.

In particular, the identity authentication may include the following:

(1) Identity Registration

A user may register an identity characteristic as follows, and a systemassociates and binds the identity characteristic registered by the userand a user identifier (a name, etc.). The registration of the identitycharacteristic includes: <1> registration of a handwritingcharacteristic: the user writes some content (such as some characters),and the system analyzes a handwriting characteristic and associates thehandwriting characteristic with other identity characteristics and theuser identifier; <2> registration of a head image characteristic: acamera of the smart whiteboard or/and the associated terminals capturesa head image of the current user one or more times, and the systemanalyzes the head image and associates the head image with otheridentity characteristics and the user identifier; <3> registration of asignature: the user signs on the whiteboard or/and the associatedterminals to recognize the association among the signature, the otheridentity characteristics, and the user identifier; <4> registration of afingerprint: the user may input a fingerprint to recognize theassociation among the fingerprint, the other identity characteristics,and the user identifier; and <5> registration of an iris characteristic:the user may collect an iris characteristic through the camera of thesmart whiteboard or/and the associated terminals to recognize theassociation among the iris characteristic, the other identitycharacteristics, and the user identifier.

(2) Identity Authentication

The smart whiteboard or/and the associated terminals accomplishes theauthentication of the current user by collecting and analyzing theidentity characteristic of the user and comparing it with an identitycharacteristic library saved in the system. The collected informationincludes: <1> handwriting: the user writes some content (e.g., somecharacters), and the system analyzes a handwriting characteristic andcompares it with an identity library to recognize the current user; <2>head image: the camera of the smart whiteboard or/and the associatedterminals captures a head image of the current user, and the systemanalyzes a head image characteristic and compares it with the identitylibrary to recognize the current user; <3> signature: the user signs onthe smart whiteboard or/and the associated terminals, and the systemcompares it with the identity library to recognize the current user; <4>fingerprint: the user inputs a fingerprint, and the system compares itwith the identity library to recognize the user; and <5> iris: thecamera of the smart whiteboard or/and the associated terminals obtainsiris information, and the system compares it with the identity libraryto recognize the current user.

3. Privilege Setting

After the identity authentication is finished, the smart whiteboardor/and associated terminals may set privileges for other users. Take ateaching scenario as an example. The teacher may set, on the smartwhiteboard, an operation privilege between the smart whiteboard and theassociated devices and an operation privilege between the associateddevices, i.e., set an operation privilege between the teacher and thestudents and an operation privilege between the students. The studentsmay set, on the associated devices, an operation privilege with otherassociated terminals, i.e., set an operation privilege between thestudents. The operation privilege includes, but is not limited to: (1)information input privilege: an input privilege for information such astext, an image, and a voice may be set for the smart whiteboard or/andassociated terminals; (2) displaying and browsing privilege: a privilegefor displaying and browsing particular content by a particular user; (3)uploading and sharing privilege: a privilege for uploading or sharingcontent written by a particular user to a sharing region; and (4)editing privilege: a privilege for editing (commenting and modifying,etc.) content of other users, etc.

Hereinafter, a detailed usage scenario is described, wherein the teacheruses the smart whiteboard and the students use the associated terminalsfor teaching.

Usage Scenario 1: The Teacher Distributes Tasks to the Students

Content to be distributed is determined in content of the smartwhiteboard, and an associated terminal receiving the content to bedistributed is determined from among the associated terminals; and inresponse to a content distribution instruction, the content to bedistributed is sent to the associated terminal receiving the content tobe distributed.

Content may be sent to one associated terminal or to multiple associatedterminals simultaneously. Or, multiple parts of content may be sent toone associated terminal or to multiple associated terminalssimultaneously.

Further, content of the smart whiteboard may be classified to generateat least one content group, and the associated terminals are classifiedto generate at least one associated terminal group. A content group isselected from the at least one content group, and an associated terminalgroup is selected from the at least one associated terminal group; and acontent distribution instruction is received, and in response to thecontent distribution instruction, content in the selected content groupis sent to each associated terminal in the selected associated terminalgroup.

In particular, as shown in FIG. 24, the teacher may list tasks formultiple users at one side of the smart whiteboard. If description ofthe tasks is relatively long, it may be displayed in the form of asummarization. Subsequently, details of the summarization may be seenvia an operation such as a gesture. For example, the teacher assigns awriting task to one student and assigns a drawing task to two students.Through operations such as a voice, a screen touch, and a gesture, theteacher may bring up information of students in the class. Theinformation such as a head image or a name may be displayed at one sideof the smart whiteboard. The teacher may select one or more students andclassify the students into groups, i.e. classify associated terminalsinto groups. Similarly, the teacher may select tasks and may classifythe tasks into groups, i.e., classify content of the smart whiteboard.The assignment of the task group on the left side to the student groupon the right side may be accomplished. At this time, the associateddevices used by the students display the assigned task.

Usage Scenario 2: The Teacher Checks and Grades a Completion Situation

The smart whiteboard obtains editing information for content provided bya designated associated terminal and sends the editing information tothe designated associated terminal.

In particular, as shown in FIG. 24, while the students respectivelyfinish their tasks, the teacher may select an image or a name of astudent on the smart whiteboard, and may display a homework completionsituation of the student (or may regard as editing information fordistributed content) in a designated area of the smart whiteboard. Theteacher may also set a time, so as to automatically update thecompletion situation of the student at the set time. When doinghomework, the teacher may forbid sharing between multiple associatedterminals. After the students finish their homework, the homework may beuploaded to the smart whiteboard. At this time, a prompt may be providedto the teacher, e.g., a voice, a shining head image, etc. If thehomework has much content, a thumbnail display may be provided. Theteacher may edit the homework, e.g., comment, correct, grade, etc. Theediting information may be synchronized with and displayed on theassociated terminal of the completer of the homework.

Usage Scenario 3: The Teacher Shares Teaching Content

Content to be shared on the smart whiteboard is determined, and thecontent to be shared is sent to a designated associated terminal.

In particular, the teacher may share a multimedia image, a video, ornotes to an associated terminal of a particular student via a gesture ora voice instruction when teaching in the class. Other students mayshare, compare, and compensate via their terminals.

Usage Scenario 4: Multiple Students Use the Same Terminal.

If the number of students is larger than that of associated terminals,multiple students may share one associated terminal. Screen spaceallocation and usage solution may be same as that when multiple usersshare the smart desktop. The teacher may designate a student via useridentity instead of an associated terminal.

Usage Scenario 5: Setting Private Information

As shown in FIG. 24, when using the smart whiteboard or/and associatedterminals, the user may perform private setting for content in adesignated area, content designated by designating an area, ordesignating content (including designating according to a summarization,selecting all, etc.). The content set as private may be hidden duringuploading and sharing, e.g., not displayed, blurred, etc.

Usage Scenario 6: Saving and Accessing of Class Notes:

After class or teaching of content is over, a student may save contentthat he has a privilege to operate. The teacher may save content ofwhole class teaching. The saved content may be accessed subsequently.The student may edit the content to generate his class notes.

Application scenario 2: multiple users use the smart desktop for ameeting, and a user saves and/or retrieves content via an associatedmobile device

The embodiments of the present disclosure may be applicable for contentsaving and retrieving. For example, after the meeting, the user may needto save the content by using the associated mobile device.

The solution may include the following:

(1) Determining an area of the smart desktop covered by the associateddevice, and transmitting content in this area of the smart desktop tothe associated device. In particular, the user may open a correspondingapplication program in the associated mobile device and put theassociated mobile device on the area of the content he wants to save. Atthis time, the smart desktop senses that the area is covered by theassociated mobile device. The content in this area may be transmitted toan application via manners such as NFC or Wi-Fi and saved in the mobileassociated terminal. Optionally, if the user wants to save all contenton the smart desktop, all content may be saved via some operations(e.g., rotating the associated mobile device on the smart desktop).

In addition, the smart desktop may receive the content transmitted bythe associated device, determine the area covered by the associateddevice, and display the received content in the determined area. Inparticular, in a meeting, the user may want to call out a record savedpreviously. The user may place the associated mobile device on a blankarea of the smart desktop where the content is to be displayed. At thistime, the record saved in the associated mobile device may betransmitted to the smart desktop via NFC, Wi-Fi, etc. and displayed inthe blank area. The user may also select the content he wants to callout and to be displayed on the smart desktop in advance.

(2) Transmitting designated content on the smart desktop to a designatedassociated device or application. In particular, after the meeting, theuser may perform a particular operation (e.g., clicking a contenttransmitting button). After detecting the user's operation, the smartdesktop transmits recording content to a designated associated device orapplication. The transmission may be implemented via a preset manner(e.g., the preset transmitting manner may include transmitting viaemail), an email address for receiving the content may be set, and thesmart desktop may transmit the recording content to the designated emailaddress. All content on the smart desktop may be transmitted to thedesignated associated device. Or, the user may designate some content onthe smart desktop and the smart desktop transmits the designated contentto the associated device. The designated associated device may be setwhen the smart desktop is initially used. Or, the associated device fromamong the associated devices may be designated to receive the contentwhen the content is transmitted.

According to an embodiment, the content may be saved in its originalformat, e.g., saving the handwritten content, or converted into a presetformat (e.g., a printed format) before saving.

Embodiment 12

User identity authentication: a user identity authentication methodprovided by the present disclosure may be applicable for the anyscenario which needs to authenticate the identity of a user of a smartdesktop in Embodiments 1 to 11, and also applicable for any identityauthentication scenario other than the smart desktop.

In the present disclosure, a user does not need to register signaturecontent in advance and does not need to write the signature content inadvance. The user is allowed to write freely to implement identityauthentication. The user authentication may be performed during a freewriting procedure of the user.

A Detailed Authentication Procedure Includes:

when a user starts to write, the smart desktop allows the user to writefreely. At this time, a system does not know identity information of theuser. The smart desktop receives written content input by the user inreal time. At this time, the smart desktop records a handwritingcharacteristic of the written content, such as a pressure, a speed, anacceleration, a stroke characteristic, etc. The smart desktop alsorecords a writing assistant characteristic, such as writing positioncoordinates, writing time information, a writing direction, a text linedirection, etc. Then, according to the handwriting characteristic of thewritten content and the writing assistant characteristic, a usercharacteristic of the current user is generated based on acharacteristic retrieving algorithm and is automatically saved in thesmart desktop. During continuous writing of the user, the smart desktopreceives content written by a user to be authenticated, and generates auser characteristic to be authenticated based on a handwritingcharacteristic of the content written by the user to be authenticatedand a writing assistant characteristic. The user characteristic isupdated in real time. At this time, the smart desktop compares the usercharacteristic to be authenticated with the saved user characteristic,and recognizes user identity from various aspects including ahandwriting characteristic, writing position coordinates, a timecontinuity, a writing direction, a text line direction, a writing area,and semantic information of the written content. If the usercharacteristic to be authenticated matches the saved usercharacteristic, the authentication of the user identity succeeds.Otherwise, a new account (user identifier) is created for the currentuser, and the writing position coordinates, time information, writingdirection, text line direction, and semantic information of the writtencontent are also recorded.

After the user identity authentication succeeds, the content written bythe user may be saved in association with the user identifier of theuser. Subsequently, it is possible to recognize which content on thesmart desktop is written by the user based on the saved user identifierand the corresponding written content.

According to an embodiment, a content displaying apparatus for a smartdesktop, applicable for a smart desktop terminal is further provided.FIG. 25 shows an exemplary structure of a content displaying apparatus2500 for a smart desktop according to some embodiments of the presentdisclosure.

As shown in FIG. 25, the content displaying apparatus 2500 includes: adisplaying instruction detecting module 2501 and a content displayingmodule 2502.

According to an embodiment, a user authentication apparatus is furtherprovided. FIG. 26 shows an exemplary structure of a user authenticationapparatus 2600 according to some embodiments of the present disclosure.The user authentication apparatus 2600 includes: a user characteristicgenerating module 2601, and an authenticating module 2602.

According to an embodiment, a smart desktop terminal is furtherprovided. FIG. 27 shows an exemplary structure of a smart desktopterminal 2700 according to some embodiments of the present disclosure.As shown in FIG. 27, the smart desktop terminal 2700 includes:

a connection establishing module 2701, adapted to establish a connectionrelationship with an associated device; and

an interacting module 2702, adapted to exchange content of a smartdesktop and/or content of the associated device with the associateddevice based on the connection relationship.

In view of the above, according to an embodiment, content may bedisplayed on the smart desktop based on a designated displaying manner,so as to meet various displaying requirements for the content of thesmart desktop.

In addition, according to an embodiment, viewing requirements of usersin different directions may be met. For example, via a thumbnail, a useris able to see content of other users which is adjusted to his viewingdirection; and via the rotation of a sharing region, each user is ableto check content in the sharing region in his suitable viewingdirection.

Further, according to an embodiment, a public sharing region isprovided, and a position and a direction of content of each user are notrestricted. Each user may write freely on the smart desktop and viewdifferent content, which meet personalized requirements of each user.

In addition, according to an embodiment, simultaneous writingrequirements of multiple users may be met, content written by each usermay be recognized simultaneously, various personalized functions such asreal time thumbnail update, summarization retrieval and update,occlusion processing, and intelligent prompt may be provided, andconvenient sharing of content written by multiple users may be realized.

According to an embodiment, the problem of “invisible” in the prior artis solved. Through a thumbnail, a subject summarization, and a usersummarization, no matter how far users and whether their viewingdirections are the same, a user is able to conveniently see content ofother users that he is interested in on a position and with an anglesuitable for him. The user may also edit the content of other users,such that multiple users may view the content more conveniently andeffectively when using the smart desktop.

Also, according to an embodiment, the problem of “unreachable” in theprior art is solved. Through content sharing manners such as sharingwith a designated user or sharing on a designated position and a sharingregion, no matter how far users and whether a writing angle is suitable,a user is able to share his content with other single or multiple usersconveniently. The shared content is automatically adjusted to a positionand angle suitable to be viewed by other users. As such, multiple usersmay share content more conveniently and effectively by using the smartdesktop.

According to an embodiment, the problem of “insufficient space forwriting” in the prior art is solved. Through partially hiding andrearranging existing content and separating new and old content, thedifficulty that a user has no writing space during a writing procedureis solved, which greatly improves the writing experience of the user onthe smart desktop.

Also, according to an embodiment, the problem of “may not move” in theprior art is solved. Through detecting a focus point of a user andrearranging a layout of occluded content and a scroll view manner forsaving screen space resources, the occlusion problem when the user viewsthe content is intelligently solved. The user is able to see the wholecontent that he is interested in conveniently without moving anoccluding object, which greatly improves the viewing experience of theuser on the smart desktop.

According to an embodiment, the problem of “may not share” in the priorart is further solved. Through controlling of recording switch andrecognition and segmentation of recording content, each user is able torecord content he is interested in freely and may conveniently andclearly check content recorded by other users and a key point of thecontent, which not only realizes recording by multiple users via amicrophone of the smart desktop but also makes usage very convenient.Through controlling to play an alarm clock sound and display a reminderin different cases, the user may be reminded based on alarm clockreminding subscribed by the user without affecting other users, whichgreatly improves the experience for using an alarm clock on the smartdesktop.

Also, according to an embodiment, the problem of “have no idea” in theprior art is solved. Through recognizing content written by each user,it is possible to intelligently provide a prompt for the user aboutcontent or a summarization he has not written but has been written byother users. Through analyzing the content written by all users, it isalso possible to correct the content written by the user intelligently,which improves the writing experience of the user, increases theefficiency for accomplishing the same task by multiple users, and avoidsrepeated conceiving and writing of the same content by multiple users.

The foregoing descriptions are only preferred embodiments of thisdisclosure and are not used for limiting the protection scope thereof.Any changes and modifications may be made by one of ordinary skill inthe art without departing from the spirit of this disclosure andtherefore should be covered within the protection scope as set by theappended claims.

FIG. 28 is a diagram illustrating a configuration for explaining a smartdesktop 2800 according to an embodiment. The smart desktop 2800 includesa processor 2810, a touch panel 2820, and a memory 2830.

The processor 2810 may control the touch panel 2820 and the memory 2830.The processor 2810 may determine content and a window displayed on thetouch panel 2820. The processor 2810 may store the content input throughthe touch panel 2820 in the memory 2830, and may read the content storedin the memory 2830.

The touch panel 2820 displays the content or the window under thecontrol of the processor 2810. The touch panel 2820 may receive a user'sinput and may transmit the user's input to the processor 2810.

The memory 2830 may store the content or user information.

FIG. 29 is a diagram for explaining changing of a position and an angleof a window according to an embodiment. The smart desktop 2800 maychange a position and an angle of the window 2910 according to a user'sposition.

The user is located at the center of the smart desktop 2800, and thesmart desktop 2800 detects the user's position and displays the window2910 in front of the user. The window 2910 displays an area allocated tothe user. The user may input content to the window 2910.

As shown in FIG. 29, the user moves to a corner of the smart desktop2800, and the smart desktop 2800 detects the user's motion, anddetermines a position and an angle of the window 2910 so that the window2910 is located in front of the user. The smart desktop 2800 displaysthe window 2910 according to the determined position and the determinedangle. Accordingly, the smart desktop 2800 may display the window 2910allocated to the user on the touch panel 2820 even when the user moves.

FIG. 30 is a diagram for explaining a method of adjusting contentaccording to an embodiment. The smart desktop 2800 may analyze contentinput by a user and may adjust a size, a gradient, and a position of thecontent.

The smart desktop 2800 may adjust characters recognized as one sentenceto the same size. As shown in FIG. 30, when sizes of characters input bythe user are gradually reduced, the smart desktop 2800 may adjust sizesof the characters to the same. The smart desktop 2800 may adjust sizesof characters included in one sentence or one paragraph to the same.

Alternatively, the smart desktop 2800 may adjust a gradient of asentence. When a sentence is inclined, the smart desktop 2800 may adjustthe gradient of the sentence in parallel to a direction of a windowallocated to the user. As shown in FIG. 30, when a gradient of asentence input by the user is gradually increased, the smart desktop2800 may adjust the gradient of the sentence to be the same as thedirection of the window.

Alternatively, the smart desktop 2800 detects a gradient of a sentenceand, when the gradient of the sentence is equal to or greater than athreshold, may adjust an angle of a window into which the sentence isinput to be the same as the gradient of the sentence. In other words,the smart desktop 2800 may adjust the angle of the window, withoutadjusting the gradient of the sentence. For example, when the gradientof the sentence is 20°, the smart desktop 2800 may adjust the angle ofthe window to 20° and may display the adjusted window on the touch panel2820.

Alternatively, the smart desktop 2800 may align a sentence. As shown inFIG. 30, when positions of first words of the sentence written atdifferent rows are different, the smart desktop 2800 may align thepositions of the first words.

FIG. 31 is a flowchart for explaining a method of displaying a thumbnailaccording to an embodiment. The smart desktop 2800 displays pieces ofcontent input by a plurality of users on thumbnails. The smart desktop2800 displays the thumbnails on windows of the users. For example, whenfirst through third users input pieces of content to the smart desktop2800, the smart desktop 2800 displays a first thumbnail on a window (afirst window) of the first user, displays a second thumbnail on a window(a second window) of the second user, and displays a third thumbnail ona window (a third window) of the third user. The first thumbnail may bedisplayed on a portion of the first window and may be enlarged anddisplayed according to the user's operation.

The smart desktop 2800 may differently display the thumbnails for theusers. For example, the first thumbnail and the second thumbnail may bedifferent from each other. When the content input by the first user tothe first window includes private information, the content input to thefirst window may not be displayed on the second thumbnail.Alternatively, the smart desktop 2800 may summarize the content input tothe first window and may display the summarized content on the secondthumbnail. Alternatively, the smart desktop 2800 may display a part ofthe content input to the first window on the second thumbnail.

In operation 3110, the processor 2810 displays windows allocated tousers on the touch panel 2820. The windows are respectively allocated tothe users. The windows indicate areas where the users may input content.

In operation 3120, the processor 2810 display pieces of content inputfrom the users on the windows.

In operation 3130, the processor 2810 displays thumbnails on thewindows. The processor 2810 displays content displayed on all windowswithin the thumbnails.

The processor 2810 designates positions of the pieces of contentaccording to positions of the users and displays the pieces of contentin the thumbnails. In other words, the processor 2810 determinesrelative positions of the users about the smart desktop 2800, anddisplays the pieces of content input by the users at positions in thethumbnails corresponding to the relative positions of the users.

The processor 2810 adjusts directions of the pieces of content displayedin the thumbnails according to directions of the windows, and displaysthe pieces of content in the thumbnails in the adjusted directions. Inother words, directions of the pieces of content input by the pluralityof users may be different from one another. The processor 2810 may alignthe pieces of content in one direction and may display the pieces ofcontent in the thumbnails so that the pieces of content input bydifferent users may be easily viewed. For example, when a first user islocated in a direction of 6 o'clock, a second user is located in adirection of 9 o'clock, a third user is located in a direction of 12o'clock, and a fourth user is located in a direction of 3 o'clock, allcontent displayed in a thumbnail of the first user may be aligned in thedirection of 6 o'clock. For example, aligning of content may be changingof an angle at which the content is displayed, changing of a size of thecontent, and changing a position where the content is displayed.

The processor 2810 omits a redundant area when the pieces of content aredisplayed in the thumbnails, and adjusts positions and sizes of thepieces of content and displays the adjusted pieces of content in thethumbnails. For example, windows may be allocated to edges of the touchpanel 2820, and the pieces of content may not be input to the center ofthe touch panel 2820. Accordingly, the processor 2810 may omit theredundant area, and may reduce a space between the pieces of content andmay arrange the pieces of content.

The processor 2810 enlarges the thumbnails into areas adjacent to thethumbnails according to inputs of the users. The thumbnails may bedisplayed in the windows, and may be enlarged and displayed in areasoutside the windows. When the users touch the thumbnails, the processor2810 may display the enlarged thumbnails. Before the thumbnails areenlarged, the pieces of content may be displayed in the form ofsummarizations in the thumbnails.

When the pieces of content are updated, the processor 2810 displays theupdated pieces of content in the thumbnails. The pieces of content inputby the users to the windows and the pieces of content displayed in thethumbnails may be synchronized.

The smart desktop 2800 may further include a sensor (not shown) fordetecting an object on the touch panel 2820. For example, the sensor maybe an infrared sensor. Alternatively, the smart desktop 2800 may detectan object by using a camera (not shown).

When the pieces of content are covered by the object, the processor 2810may rearrange and display the pieces of content on areas other than anarea occupied by the object. When the pieces of content are covered bythe object, it means that the object is located on the touch panel 2820and the user may not be able to see the pieces of content.

FIG. 32 is a flowchart for explaining a method of displaying a windowaccording to an embodiment. The smart desktop 2800 may display a windowaccording to content or a user.

In operation 3210, the processor 2810 allocates windows to a pluralityof users. The processor 2810 allocates the windows in front of the usersaccording to positions of the users. The allocating of the windows mayindicate displaying areas where the users may input content. The windowsindicate areas where the users input content and indicate portions ofthe touch panel 2820. The processor 2810 allocates the windows so thatthe windows do not overlap one another. The processor 2810 may allocatethe windows so that sizes of the windows are different from one anotheraccording to the positions of the users.

In operation 3220, the processor 2810 determines a direction and aposition of a window according to an input direction of content and amotion of a user. When the user moves to a position, the processor 2810moves the window to the position to which the user moves. For example,when the user moves rightward by 50 cm, the processor 2810 moves thewindow rightward by 50 cm.

The processor 2810 rotates the window according to a direction of theuser's body or a viewing direction. For example, when the user's body isinclined by 30° or the viewing direction is 30°, the processor 2810inclines the window by 30° and displays the inclined window. Thedirection of the body or the viewing direction may be calculated basedon edges of the smart desktop 2800.

The processor 2810 rotates the window according to the input directionof the content. The user may incline the content and input the inclinedcontent. The processor 2810 calculates a gradient of the content androtates the window by an angle corresponding to the gradient of thecontent.

In operation 3230, the processor 2810 changes the direction and theposition of the window to the determined direction and position anddisplays the window on the touch panel 2820.

When there is no input from the user for a predetermined period of time,the processor 2810 hides the window allocated to the user. Also, whenthe user is away by a predetermined distance or more from the smartdesktop 2800, the processor 2810 may hide the window.

The processor 2810 displays a sharing region at the center of the touchpanel 2820. The sharing region indicates an area where all users mayinput. After the user inputs the content to the window, the user maycopy the content and may input the content to the sharing region. Inother words, the processor 2810 displays, on the sharing region, thecontent touched by the user and dragged and dropped to the sharingregion. The sharing region is rotatable. The user may rotate the sharingregion in a desired direction by using a touch and drag operation.

The smart desktop 2800 according to an embodiment may display contentinput by all users in thumbnails.

The smart desktop 2800 according to an embodiment may change adirection, a size, and a position of a window according to an inputdirection of content or a motion of the user.

1. A content displaying method comprising: displaying windows allocatedto users on a touch panel; displaying pieces of content input from theusers on the windows; and displaying thumbnails on the windows, whereinthe displaying of the thumbnails comprises displaying pieces of contentdisplayed on all windows in the thumbnails.
 2. The content displayingmethod of claim 1, wherein the displaying of the thumbnails comprisesdisplaying the pieces of content in the thumbnails by designatingpositions of the pieces of content according to positions of the users.3. The content displaying method of claim 1, wherein the displaying ofthe thumbnails comprises displaying the pieces of content in thethumbnails by adjusting directions of the pieces of content displayed inthe thumbnails according to respective directions of the windows.
 4. Thecontent displaying method of claim 1, wherein the displaying of thethumbnails comprises omitting a redundant area when the pieces ofcontent are displayed in the thumbnails and displaying the pieces ofcontent in the thumbnails by adjusting positions and sizes of the piecesof content.
 5. The content displaying method of claim 1, furthercomprising enlarging the thumbnails into areas adjacent to thethumbnails according to inputs of the users.
 6. The content displayingmethod of claim 1, wherein the displaying of the thumbnails furthercomprises, when the pieces of content are updated, displaying theupdated pieces of content in the thumbnails.
 7. The content displayingmethod of claim 1, further comprising, when an object is detected on thetouch panel and the pieces of content are covered by the object,rearranging and displaying the pieces of content on areas other than anarea occupied by the object.
 8. A smart desktop comprising: a memory; atouch panel; and a processor configured to control the touch panel,wherein the processor is further configured to display windows allocatedto users on the touch panel, display pieces of content input from theusers on the windows, display thumbnails on the windows, and displaypieces of content displayed on all windows in the thumbnails.
 9. Thesmart desktop of claim 8, wherein the processor is further configured todisplay the pieces of content in the thumbnails by designating positionsof the pieces of content according to positions of the users.
 10. Thesmart desktop of claim 8, wherein the processor is further configured todisplay the pieces of content in the thumbnails by adjusting directionsof the pieces of content displayed in the thumbnails according torespective directions of the windows.
 11. The smart desktop of claim 8,wherein the processor is further configured to omit a redundant areawhen the pieces of content are displayed in the thumbnails and displaythe pieces of content in the thumbnails by adjusting positions and sizesof the pieces of content.
 12. The smart desktop of claim 8, wherein theprocessor is further configured to enlarge the thumbnails into areasadjacent to the thumbnails according to inputs of the users.
 13. Thesmart desktop of claim 8, wherein the processor is further configuredto, when the pieces of content are updated, display the updated piecesof content in the thumbnails.
 14. The smart desktop of claim 8, furthercomprising a sensor configured to detect an object on the touch panel,wherein the processor is further configured to, when the pieces ofcontent are covered by the object, rearrange and display the pieces ofcontent on areas other than an area occupied by the object.